Magnetometer-based gesture sensing with a wearable device

ABSTRACT

A wearable computing device such as a head-mounted display (HMD) may be equipped with a magnetometer for detecting presence and motion of a hand-wearable magnet (HWM). The HMD may analyze magnetic field measurements of the magnetometer to determine when the HWM moves within a threshold distance of the magnetometer, and may thereafter determine one or more patterns of motion of the HWM based the magnetic field measurements. The HMD may operate in a background detection state in order to determine a background magnetic field strength and to monitor for magnetic disturbances from the HWM. Upon occurrence of a trigger event corresponding to magnetic disturbance above a threshold level, the HMD may transition to operating in a gesture detection state in which it analyzes magnetometer measurements for correspondence with known gestures. Upon recognizing a known gesture, the HMD may carry out one or more actions based on the recognized known gesture.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Various technologies can be utilized to provide users with electronicaccess to data and services in communication networks, as well as tosupport communication between users. For example, devices such ascomputers, telephones, and personal digital assistants (PDAs) can beused to exchange information over communication networks including theInternet. Communication networks may in turn provide communication pathsand links to servers, which can host applications, content, and servicesthat may be accessed or utilized by users via communication devices. Thecontent can include text, video data, audio data and/or other types ofdata.

SUMMARY

In one aspect, an example embodiment presented herein provides, in awearable head-mounted display (HMD) having a processor and amagnetometer device with three orthogonal measurement axes, acomputer-implemented method comprising: operating in a backgrounddetection state; while operating in the background detection state,carrying out functions of the background state including, measuringthree orthogonal components of a background magnetic field with themagnetometer device, and determining a field magnitude of the backgroundmagnetic field from the three measured orthogonal components,determining an occurrence of a trigger from a hand-wearable magnet (HWM)at a time T_(start) upon detecting a perturbation by the HWM of thedetermined field magnitude at least as large as a perturbationthreshold, and upon determining the occurrence of the trigger,transitioning to operating in a gesture detection state; and whileoperating in the gesture detection state, carrying out functions of thegesture detection state including, tracking motion of the HWM bydetermining time derivatives of magnetic field strength measured by themagnetometer device along each of the three orthogonal measurement axes,making a comparison of the determined time derivatives with one or moresets of pre-determined time derivatives of magnetic field strength, eachof the one or more sets being stored at the wearable HMD and each beingassociated with a respective known gesture, upon matching the determinedtime derivatives with a particular set of the one or more sets based onthe comparison, identifying the respective known gesture associated withthe particular set, and transitioning to operating in the backgrounddetection state upon both of, measuring the magnitude perturbation bythe HWM of the determined field magnitude to be less than theperturbation threshold, and determining an expiration of a time intervalW that begins at T_(start).

In another aspect, an example embodiment presented herein provides awearable head-mount display (HMD) comprising: a processor; memory; amagnetometer device with three orthogonal measurement axes; means foroperating in a background detection state, wherein operating in thebackground detection state comprises carrying out functions of thebackground state including, measuring three orthogonal components of abackground magnetic field with the magnetometer device, and determininga field magnitude of the background magnetic field from the threemeasured orthogonal components, determining an occurrence of a triggerfrom a hand-wearable magnet (HWM) at a time T_(start) upon detecting aperturbation by the HWM of the determined field magnitude at least aslarge as a perturbation threshold, and upon determining the occurrenceof the trigger, transitioning to operating in a gesture detection state;and means for operating in the gesture detection state, whereinoperating in the gesture detection state comprises carrying outfunctions of the gesture detection state including, tracking motion ofthe HWM by determining time derivatives of magnetic field strengthmeasured by the magnetometer device along each of the three orthogonalmeasurement axes, making a comparison of the determined time derivativeswith one or more sets of pre-determined time derivatives of magneticfield strength, wherein each of the one or more sets is stored at thewearable HMD and each is associated with a respective known gesture,upon matching the determined time derivatives with a particular set ofthe one or more sets based on the comparison, identifying the respectiveknown gesture associated with the particular set, and transitioning tooperating in the background detection state upon both of, measuring themagnitude perturbation by the HWM of the determined field magnitude tobe less than the perturbation threshold, and determining an expirationof a time interval W that begins at T_(start).

In still another aspect, an example embodiment presented herein providesa nontransitory computer-readable medium having instructions storedthereon that, upon execution by one or more processors of a wearablehead-mounted display (HMD), cause the wearable HMD to carry outoperations comprising: operating in a background detection state; whileoperating in the background detection state, carrying out functions ofthe background state including, measuring three orthogonal components ofa background magnetic field using three orthogonal measurement axes ofmagnetometer device of the wearable HMD, and determining a fieldmagnitude of the background magnetic field from the three measuredorthogonal components, determining an occurrence of a trigger from ahand-wearable magnet (HWM) at a time T_(start) upon detecting aperturbation by the HWM of the determined field magnitude at least aslarge as a perturbation threshold, and upon determining the occurrenceof the trigger, transitioning to operating in a gesture detection state;and while operating in the gesture detection state, carrying outfunctions of the gesture detection state including, tracking motion ofthe HWM by determining time derivatives of magnetic field strengthmeasured by the magnetometer device along each of the three orthogonalmeasurement axes, making a comparison of the determined time derivativeswith one or more sets of pre-determined time derivatives of magneticfield strength, wherein each of the one or more sets is configured to bestored at the wearable HMD and each is associated with a respectiveknown gesture, upon matching the determined time derivatives with aparticular set of the one or more sets based on the comparison,identifying the respective known gesture associated with the particularset, and transitioning to operating in the background detection stateupon both of, measuring the magnitude perturbation by the HWM of thedetermined field magnitude to be less than the perturbation threshold,and determining an expiration of a time interval W that begins atT_(start).

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description, with reference where appropriate to theaccompanying drawings. Further, it should be understood that thissummary and other descriptions and figures provided herein are intendedto illustrative embodiments by way of example only and, as such, thatnumerous variations are possible. For instance, structural elements andprocess steps can be rearranged, combined, distributed, eliminated, orotherwise changed, while remaining within the scope of the embodimentsas claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 a is a first view of an example wearable head-mounted display, inaccordance with an example embodiment.

FIG. 1 b is a second view of an example wearable head-mounted display ofFIG. 1 a, in accordance with an example embodiment.

FIG. 2 is block diagram of a wearable head-mounted display, inaccordance with an example embodiment.

FIG. 3 is a simplified block diagram of a communication network, inaccordance with an example embodiment.

FIG. 4 a is a block diagram of a computing device, in accordance with anexample embodiment.

FIG. 4 b depicts a network with clusters of computing devices of thetype shown in FIG. 4 a, in accordance with an example embodiment.

FIG. 5 illustrates an example wearable head-mounted display worn by arepresentation of an example user who is depicted as also wearing ahand-wearable magnet that may be detected by a magnetometer device ofthe head-mounted display, according to an example embodiment.

FIG. 6 illustrates an example wearable head-mounted display worn by arepresentation of an example user who is depicted as also wearing ahand-wearable magnet as in FIG. 5, and further illustrates a concept ofdetection by a magnetometer device of the head-mounted display,according to an example embodiment.

FIG. 7 illustrates an example wearable head-mounted display worn by arepresentation of an example user who is depicted as also wearing ahand-wearable magnet as in FIGS. 5 and 6, and further illustrates aconcept of gesture sensing based on detection by a magnetometer deviceof the head-mounted display, according to an example embodiment.

FIG. 8 is a flowchart illustrating an example embodiment of a method ina wearable computing device for magnetometer-based gesture sensing.

FIG. 9 a depicts a state diagram including a background detection stateand gesture detection state, according to an example embodiment.

FIG. 9 a depicts a state diagram including a background detection stateand gesture recording state, according to an example embodiment.

DETAILED DESCRIPTION

1. Overview

In accordance with example embodiments, a wearable computing device mayinclude a head-mounted display (HMD) equipped with a magnetometer devicefor detecting the presence and motion of a hand-wearable magnet (HWM),such as a magnetic ring or other hand-wearable item bearing a magneticelement. Detecting the presence and motion of the HWM can be usedrecognize known patterns of motion of the HWM that correspond to knowngestures. Recognition of known patterns of motion may thereby be used toidentify of known gestures, and identification of known gestures mayprovide a basis for user input to the HMD. For example, a particulargesture could be associated with a particular command, application, orother invocable action on the HMD.

Also in accordance with example embodiments, a head-mounted display(HMD) may also include eyeglasses or goggles that can combinecomputer-generated images displayed on the eye-facing surfaces of lenselements with an actual field of view observable through the lenselements. The capability of presenting the combination of the actual,observed field-of-view (FOV) with the displayed, computer-generatedimages can be complemented or supplemented with various functions andapplications, as well as with various forms of user input and sensorydata from ancillary wearable computing components, to provide rich andvaried experiences and utility for a user or wearer of the HMD.

In example embodiments, the magnetometer device of the HMD may beconfigured to detect the presence and motion of the HWM predominantlywithin a spherical (or approximately spherical) volume of space ofapproximately arm's-length in diameter and centered (or nearly centered)at and surrounding the HMD, referred to herein as a “gesture detectionregion.” Accordingly, hand gestures detected via motion of the HWM maybe associated with the wearer of the HMD; i.e., those gestures madegenerally within the wearer's arm's length of the magnetometer-equippedHMD.

More specifically, the magnetic field strength of a small magnet orother source that may be practically configured as the magnetic elementof the HWM typically falls rapidly with distance from the source (e.g.,as r⁻³ for a dipole field for distance r), so that beyond a thresholddistance (e.g., approximately half of a meter) the HWM may not bedetectable above a threshold field strength by the magnetometer device.The gesture detection region may be seen as a spherical (orapproximately spherical) region within which the distance from the HWMto the magnetometer device is smaller than the threshold distance.Furthermore, since the field strength of the HWM falls rapidly withdistance, the boundary of the gesture detection region may be relativelysharp.

In accordance with example embodiments, the size and shape of thegesture detection region, as well as the sharpness of the boundary, maybe configured to depend on the sensitivity of the magnetometer device,and the strength and the geometry of the magnetic field of the HWM(e.g., dipole, multipole, etc.), among other possible factors. Forexample, the field strength of a dipole magnet decreases as r⁻³, whereasa quadrupole field drops off as r⁻⁴. By combining dipole and quadrupolemagnets, a field having non-spherical shape and non-isotropic decreasein field-strength can be configured so as to result in a non-sphericalgesture detection region having sharp boundaries in a well-definedregion. Other multipole fields could be used as well for similarrefining purposes. The ability to engineer the size and shape of thegesture detection region can further enhance precision, accuracy, andreliability of gesture detection and recognition.

In accordance with example embodiments, the field of the HWM may bedetected as a deviation from a background magnetic field. Moreparticularly, at any given time the magnetic field in the vicinity ofthe magnetometer device can be characterized as vector sum of abackground magnetic field (typically dominated by the Earth's magneticfield) and a possible contribution due to the magnetic field of the HWM.Beyond the gesture detection region, the magnetic field of the HWMmeasured by the magnetometer device may be much weaker than thebackground field, and consequently can be treated as a perturbation ofthe geomagnetic field. As such, the field of the HWM may be consideredto be detected if the magnetometer device measures a perturbation abovea perturbation threshold. In practice, detection of the HWM above thethreshold may be largely restricted to a region within the gesturedetection region, and correspondingly to when the HWM is located withinthe gesture detection region.

Also in accordance with example embodiments, the magnetometer device maybe configured to measure a magnetic field strength along each of threeorthogonal directions (i.e., vector components along three orthogonalaxes) in a frame of reference fixed to the magnetometer device. Amagnitude of the magnetic field can then be determined from the measuredstrength of the vector components. By continuously computing a timeaverage of the field magnitude, detection of the HWM above theperturbation threshold can be determined by comparison of the fieldmagnitude measured at a given time with the time averaged value. Sincethe boundary of the gesture detection region can be very sharp, aperturbation exceeding the threshold can occur very rapidly incomparison with the typical time duration of a gesture, and cantherefore serve as a trigger condition to signal the start time of a“gesture detection window” within which a gesture may be detected. Moreparticularly, the HMD may begin interpreting detected presence andmotion of the HWM as measured by the magnetometer device as a potentialgesture upon occurrence of a trigger condition, and may ceaseinterpreting when the trigger condition is no longer met, when theinterpreted presence and motion are identified as a known gesture, orsome combination of both.

In accordance with example embodiments, during a gesture detectionwindow, the HMD will begin acquiring time-sampled measurements of thefield strength of each of the vector components. For each vectorcomponent, a discrete time derivative can be computed as a differencebetween the field strengths of successive samples. The discrete timederivatives of the vector field, obtained from the discrete timederivatives of the vector components, may then be compared to digitallystored vector-field time derivatives associated with known (e.g.,cataloged) gestures. For example, the comparison could be based on astatistical regression analysis or on a hidden Markov model algorithm.If and when a match is determined from a comparison, the associatedgesture can then be identified as a particular known gesture. The HMDmay then carry out an action or operation associated with the particularknown gesture. If no match is found, the presence and motion of the HWMdetected during the gesture-recognition window may be considered asspurious with regard to any potential gesture.

The HMD may continue to monitor the trigger condition during the gesturedetection window. The gesture detection window may end upon the firstoccurrence of any one of (a) the trigger condition ceasing to be met,(b) successful determination of an intended gesture, or (c)determination that detected presence and motion of the HWM is spurious.The gesture detection window could also have minimum duration thatsupersedes (a) in order to ensure acquisition of a sufficient number oftime samples for gesture detection, or to avoid possibly early, falsedeterminations of trigger cessation. Once the gesture detection ends,the HMD may return to a state in which it only monitors for a triggerevent (e.g., onset of the trigger condition), but does not necessarilycompute the magnetic field time derivative or attempt to identify agesture.

In further accordance with example embodiments, stored (or cataloged)gestures may be generated as part of a “learning” procedure, whereby awearer of the HMD provides a first input to start a recording intervalfor “recording,” makes a personalized or customize hand gesture whilewearing the HWM, and then provides a second input to stop recording andend the recording interval. The recording action may thereby record thederivative of the magnetic field vectors as measured from the customizedhand gesture during the recording interval. The wearer can thereafterassociate the recorded derivatives with a particular gesture (e.g., agesture identifier, a command, an application, etc.). By repeating thisprocedure for different gestures, the wearer may build up a library orcatalog of known gestures for future use.

In still further accordance with example embodiments, the HMD will mayinclude one or more motion detectors that provide translational androtational motion measurements of the HMD. A motion detector couldinclude one or more accelerometers, one or more gyroscopes, a locationdetection system (e.g., a GPS system), some or all of which could beconfigured as an inertial measurement unit (IMU). A motion detectorcould also include a video camera, whereby analysis of video data coulddetect and measure motion in the form of optical flow. The informationfrom a motion detector can be used to compensate for HMD motion duringmagnetometer-based gesture detection. For example, if a wearer of theHMD is in a moving vehicle that is rapidly changing orientation withrespect to the geomagnetic field (e.g., moving in a circle), detectionof this motion by the motion detector could be used to disable thetrigger, and hence cause the HMD to ignore any spurious signal thatmight otherwise be taken for presence and motion of the HWM in thegesture detection region. As a further example, if a wearer of the HMDis moving his/her head while making a gesture, spurious (i.e.,non-gesture) relative motion between the HMD and the HWM could beintroduced, thereby causing failure to correctly detect a gesture.Motion detector measurements of the head motion could be used tocompensate for the spurious relative motion, and enable correct gesturedetection in this circumstance. As a still further example, intentionalmovement of the HMD with respect to a background field may be used tosignal the start of a gesture detection window.

In further accordance with example embodiments, the magnetometer devicecould include two or more three-axis magnetometers. Combiningmeasurements from multiple magnetometers could increase the precision,accuracy, and reliability of detected presence and motion of the HWM inthe gesture detection region, leading to improved performance.Furthermore, with multiple magnetometers a relative distance between theHMD and the HWM can be determined, allowing determination of atrajectory of the motion of the HWM. Such information could increase theusefulness of gestures. For instance, a virtual line could be drawnbetween two spatial points, or a virtual cursor or pointer could beinvoked as free-space gesture. Other more complex gestures could bedevised as well.

In yet further accordance with example embodiments, an HMD can becommunicatively connected with a communication network, and can exchangedata with a server or server system (other device) in the network.Multiple HMDs in a network could also exchange data. In still furtheraccordance with example embodiments, applications and/or commandsinvoked by hand gestures could involve communication with a server orserver system or with one or more other HMDs in the communicationnetwork. For example, a hand gesture could cause a program running onthe wearable HMD to upload and/or download content (e.g., media data)to/from the server or other HMD.

In accordance with example embodiments, the HWM could be in the form ofa ring, or other hand-worn jewelry. The HWM could also be in the form ofa magnetic decal affixed to one or more fingernails, or one or moremagnetized artificial fingernails. Combining a multiple-finger HWM withone or more three-axis magnetometers could allow recognition of complexgestures involving combined motions of two or more fingers, in additionto bulk hand motion. Moreover, the HWM could take the form of afashionable or stylish adornment having potential marketing value beyondits function in gesture sensing by the magnetometer device.

2. Example Systems and Network

a. Example Wearable Computing System

In accordance with an example embodiment, a wearable computing systemmay comprise various components, including one or more processors, oneor more forms of memory, one or more sensor devices, one or more I/Odevices, one or more communication devices and interfaces, and ahead-mounted display (HMD), all collectively arranged in a manner tomake the system wearable by a user. The wearable computing system mayalso include machine-language logic (e.g., software, firmware, and/orhardware instructions) stored in one or another form of memory andexecutable by one or another processor of the system in order toimplement one or more programs, tasks, applications, or the like. Thewearable computing system may be configured in various form factors,including, without limitation, integrated in the HMD as a unifiedpackage, or distributed, with one or more elements integrated in the HMDand one or more others separately wearable (e.g., as a garment, in agarment pocket, as jewelry, etc.).

Although described above as a component of a wearable computing system,it is sometimes convenient to consider an HMD to be (or at least torepresent) the wearable computing system. Accordingly, unless otherwisespecified, the terms “wearable head-mounted display” (or “wearable HMD”)or just “head-mounted display” (or “HMD”) will be used herein to referto a wearable computing system, in either an integrated (unifiedpackage) form, a distributed (or partially distributed) form, or otherwearable form.

FIG. 1 a illustrates an example wearable computing system 100 forreceiving, transmitting, and displaying data. In accordance with anexample embodiment, the wearable computing system 100 is depicted as awearable HMD taking the form of eyeglasses 102. However, it will beappreciated that other types of wearable computing devices couldadditionally or alternatively be used, including a monocular displayconfiguration having only one lens-display element.

As illustrated in FIG. 1 a, the eyeglasses 102 comprise frame elementsincluding lens-frames 104 and 106 and a center frame support 108, lenselements 110 and 112, and extending side-arms 114 and 116. The centerframe support 108 and the extending side-arms 114 and 116 are configuredto secure the eyeglasses 102 to a user's face via a user's nose andears, respectively. Each of the frame elements 104, 106, and 108 and theextending side-arms 114 and 116 may be formed of a solid structure ofplastic or metal, or may be formed of a hollow structure of similarmaterial so as to allow wiring and component interconnects to beinternally routed through the eyeglasses 102. Each of the lens elements110 and 112 may include a material on which an image or graphic can bedisplayed, either directly or by way of a reflecting surface. Inaddition, at least a portion of each lens elements 110 and 112 may besufficiently transparent to allow a user to see through the lenselement. These two features of the lens elements could be combined; forexample, to provide an augmented reality or heads-up display where theprojected image or graphic can be superimposed over or provided inconjunction with a real-world view as perceived by the user through thelens elements.

The extending side-arms 114 and 116 are each projections that extendaway from the frame elements 104 and 106, respectively, and arepositioned behind a user's ears to secure the eyeglasses 102 to theuser. The extending side-arms 114 and 116 may further secure theeyeglasses 102 to the user by extending around a rear portion of theuser's head. Additionally or alternatively, the wearable computingsystem 100 may be connected to or be integral to a head-mounted helmetstructure. Other possibilities exist as well.

The wearable computing system 100 may also include an on-board computingsystem 118, a video camera 120, a sensor 122, a finger-operable touchpad 124, and a communication interface 126. The on-board computingsystem 118 is shown to be positioned on the extending side-arm 114 ofthe eyeglasses 102; however, the on-board computing system 118 may beprovided on other parts of the eyeglasses 102. The on-board computingsystem 118 may include, for example, a one or more processors and one ormore forms of memory. The on-board computing system 118 may beconfigured to receive and analyze data from the video camera 120, thesensor 122, the finger-operable touch pad 124, and the wirelesscommunication interface 126 (and possibly from other sensory devicesand/or user interfaces) and generate images for output to the lenselements 110 and 112.

The video camera 120 is shown to be positioned on the extending side-arm114 of the eyeglasses 102; however, the video camera 120 may be providedon other parts of the eyeglasses 102. The video camera 120 may beconfigured to capture images at various resolutions or at differentframe rates. Video cameras with a small form factor, such as those usedin cell phones or webcams, for example, may be incorporated into anexample of the wearable system 100. Although FIG. 1 a illustrates onevideo camera 120, more video cameras may be used, and each may beconfigured to capture the same view, or to capture different views. Forexample, the video camera 120 may be forward facing to capture at leasta portion of a real-world view perceived by the user. This forwardfacing image captured by the video camera 120 may then be used togenerate an augmented reality where computer generated images appear tointeract with the real-world view perceived by the user.

The sensor 122 may be used to measure and/or determine location,orientation, and motion information, for example. Although representedas a single component mounted on the extending side-arm 116 of theeyeglasses 102, the sensor 122 could in practice include more than onetype of sensor device or element provided on one or more different partsof the eyeglasses 102.

By way of example and without limitation, the sensor 122 could includeone or more of motion detectors (e.g., one or more gyroscopes and/oraccelerometers), one or more magnetometers, and a location determinationdevice (e.g., a GPS device). Gyroscopes, accelerometers, andmagnetometers may be integrated into what is conventionally called an“inertial measurement unit” (IMU). An IMU may, in turn, be part of an“attitude heading reference system” (AHRS) that computes (e.g., usingthe on-board computing system 118) a pointing direction of the HMD fromIMU sensor data, possibly together with location information (e.g., froma GPS device). Accordingly, the sensor 122 could include or be part ofan AHRS. Other sensing devices or elements may be included within thesensor 122 and other sensing functions may be performed by the sensor122. Additional details regarding magnetometers and magnetometeroperation in connection with gesture detection and recognition arediscussed below.

The finger-operable touch pad 124, shown mounted on the extendingside-arm 114 of the eyeglasses 102, may be used by a user to inputcommands. The finger-operable touch pad 124 may sense at least one of aposition and a movement of a finger via capacitive sensing, resistancesensing, or a surface acoustic wave process, among other possibilities.The finger-operable touch pad 124 may be capable of sensing fingermovement in a direction parallel to the pad surface, in a directionnormal to the pad surface, or both, and may also be capable of sensing alevel of pressure applied. The finger-operable touch pad 124 may beformed of one or more translucent or transparent insulating layers andone or more translucent or transparent conducting layers. Edges of thefinger-operable touch pad 124 may be formed to have a raised, indented,or roughened surface, so as to provide tactile feedback to a user whenthe user's finger reaches the edge of the finger-operable touch pad 124.Although not shown in FIG. 1 a, the eyeglasses 102 could include onemore additional finger-operable touch pads, for example attached to theextending side-arm 316, which could be operated independently of thefinger-operable touch pad 124 to provide a duplicate and/or differentfunction.

The communication interface 126 could include an antenna and transceiverdevice for support of wireline and/or wireless communications betweenthe wearable computing system 100 and a remote device or communicationnetwork. For instance, the communication interface 126 could supportwireless communications with any or all of 3G and/or 4G cellular radiotechnologies (e.g., CDMA, EVDO, GSM, UMTS, LTE, WiMAX), as well aswireless local or personal area network technologies such as aBluetooth, Zigbee, and WiFi (e.g., 802.11a, 802.11b, 802.11g). Othertypes of wireless access technologies could be supported as well. Thecommunication interface 126 could enable communications between thewearable computing system 100 and one or more end devices, such asanother wireless communication device (e.g., a cellular phone or anotherwearable computing device), a user at a computer in a communicationnetwork, or a server or server system in a communication network. Thecommunication interface 126 could also support wired accesscommunications with Ethernet or USB connections, for example.

FIG. 1 b illustrates another view of the wearable computing system 100of FIG. 1 a. As shown in FIG. 1 b, the lens elements 110 and 112 may actas display elements. In this regard, the eyeglasses 102 may include afirst projector 128 coupled to an inside surface of the extendingside-arm 116 and configured to project a display image 132 onto aninside surface of the lens element 112. Additionally or alternatively, asecond projector 130 may be coupled to an inside surface of theextending side-arm 114 and configured to project a display image 134onto an inside surface of the lens element 110.

The lens elements 110 and 112 may act as a combiner in a lightprojection system and may include a coating that reflects the lightprojected onto them from the projectors 128 and 130. Alternatively, theprojectors 128 and 130 could be scanning laser devices that interactdirectly with the user's retinas.

A forward viewing field may be seen concurrently through lens elements110 and 112 with projected or displayed images (such as display images132 and 134). This is represented in FIG. 1 b by the field of view (FOV)object 136-L in the left lens element 112 and the same FOV object 136-Rin the right lens element 110. The combination of displayed images andreal objects observed in the FOV may be one aspect of augmented reality,referenced above. In addition, images could be generated for the rightand left lens elements produce a virtual three-dimensional space whenright and left images are synthesized together by a wearer of the HMD.Virtual objects could then be made to appear to be located in and occupythe actual three-dimensional space viewed transparently through thelenses.

In alternative embodiments, other types of display elements may also beused. For example, lens elements 110, 112 may include: a transparent orsemi-transparent matrix display, such as an electroluminescent displayor a liquid crystal display; one or more waveguides for delivering animage to the user's eyes; and/or other optical elements capable ofdelivering an in focus near-to-eye image to the user. A correspondingdisplay driver may be disposed within the frame elements 104 and 106 fordriving such a matrix display. Alternatively or additionally, a scanninglaser device, such as low-power laser or LED source and accompanyingscanning system, can draw a raster display directly onto the retina ofone or more of the user's eyes. The user can then perceive the rasterdisplay based on the light reaching the retina.

Although not shown in FIGS. 1 a and 1 b, the wearable system 100 canalso include one or more components for audio output. For example,wearable computing system 100 can be equipped with speaker(s),earphone(s), and/or earphone jack(s). Other possibilities exist as well.

While the wearable computing system 100 of the example embodimentillustrated in FIGS. 1 a and 1 b is configured as a unified package,integrated in the HMD component, other configurations are possible aswell. For example, although not explicitly shown in FIGS. 1 a and 1 b,the wearable computing system 100 could be implemented in a distributedarchitecture in which all or part of the on-board computing system 118is configured remotely from the eyeglasses 102. For example, some or allof the on-board computing system 118 could be made wearable in or onclothing as an accessory, such as in a garment pocket or on a belt clip.Similarly, other components depicted in FIGS. 1 a and/or 1 b asintegrated in the eyeglasses 102 could also be configured remotely fromthe HMD component. In such a distributed architecture, certaincomponents might still be integrated in HMD component. For instance, oneor more sensors (e.g., a magnetometer, gyroscope, etc.) could beintegrated in eyeglasses 102.

In an example distributed configuration, the HMD component (includingother integrated components) could communicate with remote componentsvia the communication interface 126 (or via a dedicated connection,distinct from the communication interface 126). By way of example, awired (e.g. USB or Ethernet) or wireless (e.g., WiFi or Bluetooth)connection could support communications between a remote computingsystem and a HMD component. Additionally, such a communication linkcould be implemented between a HMD component and other remote devices,such as a laptop computer or a mobile telephone, for instance.

FIG. 2 is a block diagram depicting functional components of an examplewearable computing system 202 in accordance with an example embodiment.As shown in FIG. 2, the example wearable computing system 202 includesone or more processing units 204, data storage 206, transceivers 212,communication interfaces 214, user input/output (I/O) devices 216, andsensor devices 228, all of which may be coupled together by a system bus238 or other communicative interconnection means. These components maybe arranged to support operation in accordance with an exampleembodiment of a wearable computing system, such as system 100 shown inFIGS. 1 a and 1 b, or other wearable HMD.

The one or more processing units 204 could include one or moregeneral-purpose processors (e.g., INTEL microprocessors) and/or one ormore special-purpose processors (e.g., dedicated digital signalprocessor, application specific integrated circuit, etc.). In turn, thedata storage 206 could include one or more volatile and/or non-volatilestorage components, such as magnetic or optical memory or disk storage.Data storage 206 can be integrated in whole or in part with processingunit 204, as cache memory or registers for instance. As further shown,data storage 206 is equipped to hold program logic 208 and program data210.

Program logic 208 could include machine language instructions (e.g.,software code, firmware code, etc.) that define routines executable bythe one or more processing units 204 to carry out various functionsdescribed herein. Program data 210 could contain data used ormanipulated by one or more applications or programs executable by theone or more processors. Such data can include, among other forms ofdata, program-specific data, user data, input/output data, sensor data,or other data and information received, stored, retrieved, transmitted,analyzed, or modified in the course of execution of one or more programsor applications.

The transceivers 212 and communication interfaces 214 may be configuredto support communication between the wearable computing system 202 andone or more end devices, such as another wireless communication device(e.g., a cellular phone or another wearable computing device), a user ata computer in a communication network, or a server or server system in acommunication network. The transceivers 212 may be coupled with one ormore antennas to enable wireless communications, for example, asdescribe above for the wireless communication interface 126 shown inFIG. 1 a. The transceivers 212 may also be coupled with one or more andwireline connectors for wireline communications such as Ethernet or USB.The transceivers 212 and communication interfaces 214 could also be usedsupport communications within a distributed-architecture in whichvarious components of the wearable computing system 202 are locatedremotely from one another. In this sense, the system bus 238 couldinclude elements and/or segments that support communication between suchdistributed components.

As shown, the user I/O devices 216 include a camera 218, a display 220,a speaker 222, a microphone 224, and a touchpad 226. The camera 218could correspond to the video camera 120 described in the discussion ofFIG. 1 a above. Similarly, the display 220 could correspond to an imageprocessing and display system for making images viewable to a user(wearer) of an HMD. The display 220 could include, among other elements,the first and second projectors 128 and 130 coupled with lens elements112 and 110, respectively, for generating image displays as describedabove for FIG. 1 b. The touchpad 226 could correspond to thefinger-operable touch pad 124, as described for FIG. 1 a. The speaker422 and microphone 224 could similarly correspond to componentsreferenced in the discussion above of FIGS. 1 a and 1 b. Each of theuser I/O devices 216 could also include a device controller and stored,executable logic instructions, as well as an interface for communicationvia the system bus 238.

The sensor devices 228, which could correspond to the sensor 122described above for FIG. 1 a, include a location sensor 230, a motionsensor 232, one or more magnetometers 234, and an orientation sensor236. The location sensor 230 could correspond to a Global PositioningSystem (GPS) device, or other location-determination device (e.g. mobilephone system triangulation device, etc.). The motion sensor 232 couldcorrespond to one or more accelerometers and/or one or more gyroscopes.A typical configuration may include three accelerometers oriented alongthree mutually orthogonal axes, for example. A similar configuration ofthree magnetometers can also be used.

The orientation sensor 236 could include or be part of an AHRS forproviding theodolite-like functionality for determining an angularorientation of a reference pointing direction of the HMD with respect toa local terrestrial coordinate system. For instance, the orientationsensor could determine an altitude angle with respect to horizontal andan azimuth angle with respect to a reference directions, such asgeographic (or geodetic) North, of a forward pointing direction of theHMD. Other angles and coordinate systems could be used as well fordetermining orientation.

The magnetometer 234 (or magnetometers) could be used to determine thestrength and direction of the Earth's magnetic (geomagnetic) field asmeasured at a current location of the HMD. As described in detail below,the magnetometer (or magnetometers) could also be used to detectpresence and motion of a hand-wearable magnet in order to recognize handgestures.

Each of the sensor devices 228 could also include a device controllerand stored, executable logic instructions, as well as an interface forcommunication via the system bus 238.

It will be appreciated that there can be numerous specificimplementations of a wearable computing system or wearable HMD, such asthe wearable computing system 202 illustrated in FIG. 2. Further, one ofskill in the art would understand how to devise and build such animplementation.

b. Example Network

In an example embodiment, an HMD can support communications with anetwork and with devices in or communicatively connected with a network.Such communications can include exchange of information between the HMDand another device, such as another connected HMD, a mobile computingdevice (e.g., mobile phone or smart phone), or a server. Informationexchange can support or be part of services and/or applications,including, without limitation, uploading and/or downloading content(e.g., music, video, etc.), and client-server communications, amongothers.

FIG. 3 illustrates one view of a network 300 in which one or more HMDscould engage in communications. As depicted, the network 300 includes adata network 302 that is connected to each of a radio access network(RAN) 304, a wireless access network 306, and a wired access network308. The data network 302 could represent the one or more interconnectedcommunication networks, such as or including the Internet. The radioaccess network 304 could represent a service provider's cellular radionetwork supporting, for instance, 3G and/or 4G cellular radiotechnologies (e.g., CDMA, EVDO, GSM, UMTS, LTE, WiMAX). The wirelessaccess network 306 could represent a residential or hot-spot wirelessarea network supporting, such as, Bluetooth, ZigBee, and WiFi (e.g.,802.11a, 802.11b, 802.11g). The wired access network 308 could representa residential or commercial local area network supporting, for instance,Ethernet.

The network 300 also includes a server system 310 connected to the datanetwork 302. The server system 310 could represent a website or othernetwork-based facility for providing one or another type of service tousers. For instance, in accordance with an example embodiment, theserver system 310 could host an online social networking service orwebsite. As another example, the server system 310 could provide anetwork-based information search service.

FIG. 3 also shows various end-user and/or client devices connected tothe network 300 via one of the three access networks. By way of example,an HMD 312 is connected to the RAN 304 via an air interface 313 (e.g., a3G or 4G technology), and an HMD 314 is connected to the RAN 304 via anair interface 315 (e.g., a 3G or 4G technology). Also by way of example,an HMD 316 is connected to the wireless access network 306 via an airinterface 317 (e.g., a WiFi technology). In addition and also by way ofexample, a mobile phone 318 is shown connected to the RAN 304 via an airinterface 319, a smart phone 320 is shown connected to the wirelessaccess network 306 via an air interface 321, and a laptop computer 322is shown connected to the wired access network 308 via a wired interface323. Each of the end-user devices could communicate with one or anothernetwork-connected device via its respective connection with the network.It could be possible as well for some of these end-user devices tocommunicate directly with each other (or other end-user devices notshown).

Each of the HMDs 312, 314, and 316 is depicted as being worn bydifferent user (each user being represented by a cartoon face) in orderto signify possible user-related variables, circumstances, andapplications that may be associated with each HMD. For instance, the HMD312 could at one time upload content to an online social networkingservice, whereas the HMD 314 could at the same or another time send arequest to a network-based information search service. Users couldinteract with each other and/or with the network via their respectiveHMDs, and in turn use magnetically-sensed hand gestures as one ofmultiple possible user interfaces with their respective HMDs. Otherexamples are possible as well. For the purposes of most of thediscussion herein it is usually sufficient to reference only an HMDwithout referencing the user (or wearer) of the HMD. Explicit referenceto or discussion of a user (or wearer) of an HMD will be made asnecessary.

c. Example Server System

A network server, such as the server system 310 in FIG. 3, could takevarious forms and be implemented in one or more different ways. FIGS. 4a and 4 b illustrate two example embodiments of a server system: anintegrated system including a representative computing device (FIG. 4a), and a distributed system (FIG. 4 b) including multiplerepresentative computing devices, as well as additional system elements,communicatively connected together.

FIG. 4 a is a block diagram of a computing device 400 in accordance withan example embodiment. The computing device 400 can include a userinterface module 401, a network-communication interface module 402, oneor more processors 403, and data storage 404, all of which can be linkedtogether via a system bus, network, or other connection mechanism 405.

The user interface module 401 can be operable to send data to and/orreceive data from external user input/output devices. For example, theuser interface module 401 can be configured to send/receive data to/fromuser input devices such as a keyboard, a keypad, a touch screen, acomputer mouse, a track ball, a joystick, and/or other similar devices,now known or later developed. The user interface module 401 can also beconfigured to provide output to user display devices, such as one ormore cathode ray tubes (CRT), liquid crystal displays (LCD), lightemitting diodes (LEDs), displays using digital light processing (DLP)technology, printers, light bulbs, and/or other similar devices, nowknown or later developed. The user interface module 401 can also beconfigured to generate audible output(s), such as a speaker, speakerjack, audio output port, audio output device, earphones, and/or othersimilar devices, now known or later developed.

The network-communications interface module 402 can include one or morewireless interfaces 407 and/or wireline interfaces 408 that areconfigurable to communicate via a network, such as the network 302 shownin FIG. 3. The wireless interfaces 407 can include one or more wirelesstransceivers, such as a Bluetooth transceiver, a Wi-Fi transceiverperhaps operating in accordance with an IEEE 802.11 standard (e.g.,802.11a, 802.11b, 802.11g), a WiMAX transceiver perhaps operating inaccordance with an IEEE 802.16 standard, and/or other types of wirelesstransceivers configurable to communicate via a wireless network. Thewireline interfaces 408 can include one or more wireline transceivers,such as an Ethernet transceiver, a Universal Serial Bus (USB)transceiver, or similar transceiver configurable to communicate via awire, a twisted pair of wires, a coaxial cable, an optical link, afiber-optic link, or other physical connection to a wireline network.

In some embodiments, the network communications interface module 402 canbe configured to provide reliable, secured, compressed, and/orauthenticated communications. For each communication described herein,information for ensuring reliable communications (e.g., guaranteedmessage delivery) can be provided, perhaps as part of a message headerand/or footer (e.g., packet/message sequencing information,encapsulation header(s) and/or footer(s), size/time information, andtransmission verification information such as cyclic redundancy check(CRC) and/or parity check values). Communications can be compressed anddecompressed using one or more compression and/or decompressionalgorithms and/or protocols such as, but not limited to, one or morelossless data compression algorithms and/or one or more lossy datacompression algorithms. Communications can be made secure (e.g., beencoded or encrypted) and/or decrypted/decoded using one or morecryptographic protocols and/or algorithms, such as, but not limited to,DES, AES, RSA, Diffie-Hellman, and/or DSA. Other cryptographic protocolsand/or algorithms can be used as well or in addition to those listedherein to secure (and then decrypt/decode) communications.

The one or more processors 403 can include one or more general purposeprocessors and/or one or more special purpose processors (e.g., digitalsignal processors, application specific integrated circuits, etc.). Theone or more processors 403 can be configured to executecomputer-readable program instructions 406 that are contained in thedata storage 404 and/or other instructions as described herein.

The data storage 404 can include one or more computer-readable storagemedia that can be read or accessed by at least one of the processors403. The one or more computer-readable storage media can includevolatile and/or non-volatile storage components, such as optical,magnetic, organic or other memory or disc storage, which can beintegrated in whole or in part with at least one of the one or moreprocessors 403. In some embodiments, the data storage 404 can beimplemented using a single physical device (e.g., one optical, magnetic,organic or other memory or disc storage unit), while in otherembodiments, the data storage 404 can be implemented using two or morephysical devices.

Computer-readable storage media associated with data storage 404 and/orother computer-readable media described herein can also includenon-transitory computer-readable media such as computer-readable mediathat stores data for short periods of time like register memory,processor cache, and random access memory (RAM). Computer-readablestorage media associated with data storage 404 and/or othercomputer-readable media described herein can also include non-transitorycomputer readable media that stores program code and/or data for longerperiods of time, such as secondary or persistent long term storage, likeread only memory (ROM), optical or magnetic disks, compact-disc readonly memory (CD-ROM), for example. Computer-readable storage mediaassociated with data storage 404 and/or other computer-readable mediadescribed herein can also be any other volatile or non-volatile storagesystems. Computer-readable storage media associated with data storage404 and/or other computer-readable media described herein can beconsidered computer readable storage media for example, or a tangiblestorage device.

The data storage 404 can include computer-readable program instructions406 and perhaps additional data. In some embodiments, the data storage404 can additionally include storage required to perform at least partof the herein-described techniques, methods, and/or at least part of thefunctionality of the herein-described devices and networks.

FIG. 4 b depicts a network 406 with computing clusters 409 a, 409 b, and409 c in accordance with an example embodiment. In FIG. 4 b, functionsof a network server, such as the server system 310 in FIG. 3, can bedistributed among three computing clusters 409 a, 409 b, and 408 c. Thecomputing cluster 409 a can include one or more computing devices 400 a,cluster storage arrays 410 a, and cluster routers 411 a, connectedtogether by local cluster network 412 a. Similarly, computing cluster409 b can include one or more computing devices 400 b, cluster storagearrays 410 b, and cluster routers 411 b, connected together by localcluster network 412 b. Likewise, computing cluster 409 c can include oneor more computing devices 400 c, cluster storage arrays 410 c, andcluster routers 411 c, connected together by a local cluster network 412c.

In some embodiments, each of computing clusters 409 a, 409 b, and 409 ccan have an equal number of computing devices, an equal number ofcluster storage arrays, and an equal number of cluster routers. In otherembodiments, however, some or all of computing clusters 409 a, 409 b,and 409 c can have different numbers of computing devices, differentnumbers of cluster storage arrays, and/or different numbers of clusterrouters. The number of computing devices, cluster storage arrays, andcluster routers in each computing cluster can depend on the computingtask or tasks assigned to each computing cluster.

Cluster storage arrays 410 a, 410 b, and 410 c of computing clusters 409a, 409 b, and 409 c can be data storage arrays that include disk arraycontrollers configured to manage read and write access to groups of harddisk drives. The disk array controllers, alone or in conjunction withtheir respective computing devices, can also be configured to managebackup or redundant copies of the data stored in the cluster storagearrays to protect against disk drive or other cluster storage arrayfailures and/or network failures that prevent one or more computingdevices from accessing one or more cluster storage arrays.

The cluster routers 411 a, 411 b, and 411 c in the computing clusters409 a, 409 b, and 409 c can include networking equipment configured toprovide internal and external communications for the computing clusters.For example, the cluster routers 411 a in the computing cluster 409 acan include one or more internet switching and/or routing devicesconfigured to provide (i) local area network communications between thecomputing devices 400 a and the cluster storage arrays 401 a via thelocal cluster network 412 a, and/or (ii) wide area networkcommunications between the computing cluster 409 a and the computingclusters 409 b and 409 c via the wide area network connection 413 a tothe network 406. The cluster routers 411 b and 411 c can include networkequipment similar to the cluster routers 411 a, and the cluster routers411 b and 411 c can perform similar networking functions for thecomputing clusters 409 b and 409 b that the cluster routers 411 aperform for the computing cluster 409 a.

3. Gesture Recognition with a Magnetometer Device

Example embodiments of magnetometer-based gesture sensing with amagnetometer-equipped HMD may be described in terms of operation byconsidering representative use of an example HMD and an examplehand-wearable magnet (HWM). For purposes of illustration, a HMD, such asthe wearable computing device 100 of FIG. 1, that includes amagnetometer device, as well as a HWM, may be taken as being worn byrepresentative user. Also for illustrative purposes, the HWM may betaken to be a magnetic ring or a ring bearing a magnetic element. By wayof example, the magnetic element (or ring) may be considered a dipolemagnet, unless otherwise specified. It will be appreciated thathand-wearable magnets with different form factors may also be used (e.g,other jewelry), and that magnetic elements with higher-order multipolesmay be employed. In illustrative operation, the HWM can be used by therepresentative user to facilitate user interaction with the HMD, asdescribed below.

In accordance with example embodiments, a magnetometer device mayinclude a magnetometer for measuring local magnetic field. Themagnetometer device could also include additional components, such as apower supply, an interface for connecting and communicating with theHMD, and a processor for carrying out various computational functions(such as ones discussed below) and/or control functions (such asdetermining modes of operation, setting adjustable parameters, etc.).Alternatively, these and possibly other, additional components andfunctions could be integrated with the magnetometer, in which case themagnetometer and the magnetometer device could be considered one and thesame. In still an alternative configuration, the magnetometer devicecould include more than one magnetometer, or multiplesingle-magnetometer devices could be used, as discussed later in moredetail.

a. Operational States

In accordance with example embodiments, the magnetometer of a HMD maycontinuously monitor and measure a local magnetic field, and the HMDcould recognize a particular gesture of a hand bearing a HWM byanalyzing the effect of gesture-related motion of the HWM on the localmagnetic field measured by the magnetometer. In typical usage scenarios,hand gestures may be carried out as transient actions within a localized“gesture detection region” of space near the HMD (e.g., within aspherical volume of roughly an arm's length radius, centered at theHMD). Beyond the gesture detection region, the HWM may be largelyundetectable by the magnetometer. Hence, in the context of detection andmeasurement of the HWM by the magnetometer, hand gestures may be viewedas transient magnetic disturbances against a background magnetic field.Accordingly, the HMD may be configured to detect and respond to suchtransient magnetic disturbances.

In further accordance with example embodiments, the HMD may beconfigured to operate in different states with respect tomagnetometer-based gesture recognition. More particularly, the HMD mayoperate in a background detection state in order to determine an initialbaseline level of local magnetic field strength. While operating in thebackground detection state, the HMD may also anticipate hand gestures bymonitoring for magnetic disturbances attributable to presence and motionof the HWM within a threshold distance of the magnetometer. Uponoccurrence of a trigger event corresponding to magnetic disturbanceabove a threshold level, the HMD may then transition to operating in agesture detection state. While operating in the gesture detection state,the HMD may analyze measurements from the magnetometer in order todetermine if they correspond to a known gesture. If so, the HMD may thencarry out one or more actions based on the now-recognized known gesture.For example, the HMD could carry out a user command associated with theknown gesture.

As described below, analysis of magnetometer measurements duringoperation in the gesture detection state may include comparison of theanalyzed measurements with stored information associated with knowngestures. In further accordance with example embodiments, the storedinformation may be generated by way of a “gesture learning” procedurefor creating new, known gestures, and recording them on the HMD (e.g.,in program data 208 illustrated in FIG. 2). Also in accordance withexample embodiments, the HMD may further be configured to operate in agesture recording state, during which the gesture learning procedure maybe carried out.

Details of the background detection state, the gesture detection state,and the gesture recording state, as well as of conditions fortransitions between states, are discussed below.

b. Magnetometer Measurement and Operation

In accordance with example embodiments, the magnetometer may be athree-axis magnetometer for measuring local magnetic field along each ofthree orthogonal axes. For brevity, the term “three-axis” will droppedin the following discussion, but will be understood to apply to the term“magnetometer” hereinafter unless otherwise indicated. By way ofexample, the three orthogonal axes will be taken to be {circumflex over(x)}, ŷ, and {circumflex over (z)} of a rectangular coordinate systemreferenced to (i.e., fixed to) the magnetometer. As such, themagnetometer can be configured to measure a magnetic field vector {rightarrow over (H)}=[H_(x), H_(y), H_(z)], where H_(x), H_(y), and H_(z) arethe vector components of {right arrow over (H)} along 2, y, and 2,respectively. For any given measurement, a magnitude h of the magneticfield (i.e., magnetic field “strength”) can be calculated as:

$\begin{matrix}{{h \equiv {\overset{\rightarrow}{H}}} = {\sqrt{H_{x}^{2} + H_{y}^{2} + H_{z}^{2}}.}} & (1)\end{matrix}$

In further accordance with example embodiments, the magnetometer may beconfigured to measure magnetic field in successive samples at discretesampling times t_(i), i=1, 2, . . . , M, where for the moment, M is anarbitrary integer. In notational terms, {right arrow over(H)}_(i)=[(H_(x)), (H_(y)), (H_(z))_(i)], where (H_(x))_(i),(H_(y))_(i), and (H_(z))_(i), are the vector components of the fieldsampled at t_(i), i=1, 2, . . . , M. A field magnitude h_(i) may bedetermined for each sample by applying equation (1) at each sample timet_(i), i=1, 2, . . . , M.

In still further accordance with example embodiments, the discretesampling times t_(i) are regular, such that Δt=t_(i−1)−t_(i) is the samefor all i. Conventionally, Δt is referred to as the sampling interval,and 1/Δt is referred to as the sampling frequency. In accordance withexample embodiments, sampling intervals in the range 10-20 msec(0.01-0.02 seconds) may be used, corresponding to sampling frequenciesin the range 50-100 Hz (1 Hz=1 cycle per second). It will be appreciatedthat sampling intervals outside of this range are possible as well. Inpractice, each sample may correspond to an accumulation (e.g., a timeaverage) of a continuous measurement of {right arrow over (H)}_(i) overthe duration of Δt.

Also in accordance with example embodiments, the magnetometer could beconfigured to provide each {right arrow over (H)}_(i) sample to the HMD(e.g., via an integrated interface or an interface component of themagnetometer device), which in turn could compute each h_(i).Alternatively, the magnetometer (or magnetometer device) could includefunctionality to compute each h_(i). Additional functions of themagnetometer and/or magnetometer device could include adjustment of Δtand/or other operational parameters.

c. Background Determination and Background Detection State

In typical operational settings of a HMD (e.g, as worn by arepresentative user), the primary source of background magnetic fieldwill be the Earth's magnetic field (geomagnetic field). While thedirection of the geomagnetic field measured by the magnetometer willdepend on the orientation of magnetometer (i.e., of the threemeasurement axes {circumflex over (x)}, ŷ, and {circumflex over (z)})with respect to a coordinate system fixed to the Earth, the magnitude ofthe geomagnetic field at a given location does not. In addition, themagnitude of the geomagnetic field generally varies sufficiently slowlywith location that it may be considered nearly constant over roughlymetropolitan-size regions. Local sources of magnetic field or ofmagnetic field disturbances (e.g., electrical wires and metalstructures) may also contribute to the background, but these generallytend to be weak, contributing mainly as noise to an average backgroundlevel.

In accordance with example embodiments, the background magnetic fieldcan be determined as a time average of successive samples of fieldmagnitude h_(i). For N samples at discrete sample times t_(i), i=1, . .. , N, the sample mean field strength μ is given by:

$\begin{matrix}{\mu = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{h_{i}.}}}} & \left( {2\; a} \right)\end{matrix}$More generally, for N discrete times t_(i), i=N₁, . . . , N₂, over asliding time window W_(N) ₁ _(,N) ₂ from N₁ to N₂, where N₂=N₁+N−1, andN≧1, the sample mean field strength μ_(N) ₁ _(,N) ₂ is given by:

$\begin{matrix}{\mu_{N_{1},N_{2}} = {\frac{1}{N}{\sum\limits_{i = N_{1}}^{N_{2}}{h_{i}.}}}} & \left( {2b} \right)\end{matrix}$The sliding window W_(N) ₁ _(,N) ₂ corresponds to incrementing both N₁and N₂ by k for every k increments of Δt in time. For example, for k=1,W_(N) ₁ _(,N) ₂ slides Δt toward increasing time for each Δt of passingtime. This gives an adaptive form of the sample mean that updates withincreasing time.

It is sometimes convenient to adopt an alternate notation in which thediscrete sample times are t_(i), i=0, . . . , N−1, where t₀ is taken tobe the current time, and in which each increment in the index icorresponds to the next previous (earlier) sample time. In this case, asliding time window W_(0,N) is referenced to the current time and coversN consecutive samples backwards in time starting from t₀. For example,the sample mean at the current time t₀ would be given by:

$\begin{matrix}{\mu_{0} = {{\frac{1}{N}\left( {h_{0} + h_{- 1} + \ldots + h_{1 - N}} \right)} = {\frac{1}{N}{\sum\limits_{i = 0}^{N - 1}{h_{- i}.}}}}} & \left( {2c} \right)\end{matrix}$

A standard deviation σ of the sample mean μ in equation (2a) may becomputed as:

$\begin{matrix}{\sigma = {\sqrt{\frac{1}{N - 1}{\sum\limits_{i = 1}^{N}\left( {h_{i} - \mu} \right)^{2}}}.}} & \left( {3a} \right)\end{matrix}$

For the more general case of the sliding time window W_(N) ₁ _(,N) ₂ ,an adaptive form of the standard deviation σ_(N) ₁ _(,N) ₂ of the samplemean ρ_(N) ₁ _(,N) ₂ in equation (2b) may be computed as:

$\begin{matrix}{\sigma_{N_{1},N_{2}} = {\sqrt{\frac{1}{N - 1}{\sum\limits_{i = N_{1}}^{N_{2}}\left( {h_{i} - \mu_{N_{1},N_{2}}} \right)^{2}}}.}} & \left( {3b} \right)\end{matrix}$In terms of W_(0,N), the standard deviation σ₀ at t₀ may be computed as:

$\begin{matrix}{\sigma_{0} = {\sqrt{\frac{1}{N - 1}{\sum\limits_{i = 0}^{N - 1}\left( {h_{- i} - \mu_{0}} \right)^{2}}}.}} & \left( {3c} \right)\end{matrix}$

In further accordance with example embodiments, while operating in thebackground detection state, the HMD can compute h_(i) from the samplesof {right arrow over (H)}_(i) measured by the magnetometer at timest_(i). The HMD may then compute the sample mean and standard deviationof the field strength (magnitude). More particularly, baseline values ofsample mean and standard deviation may be established during acalibration period by acquiring an initial set of M measurements atconsecutive times t_(i) when the HWM is known to not be detected above athreshold level. For example, a user could hold the HWM beyond thethreshold distance during the calibration period.

In practice, the accuracy of the baseline values of sample mean andstandard deviation may be improved by adopting a calibration period thatis long compared with known or expected short-term (e.g., tens ofseconds) field disturbances. This corresponds to M>>1. For example,taking Δt=10 msec, a five-minute calibration period corresponds toM=30,000. During this time, the HMD may be moved about to a number ofdifferent locations and oriented in a number of different orientationswithin a region of expected use. As an example, a user might walk arounda room or neighborhood during the calibration period. Note that aprecise value of M need not necessarily be specified, so long as M>>1can be reasonably understood to be satisfied. Thus, a calibration periodof “approximately X” minutes could be an adequate specification, so longas “approximately X minutes” could be reasonably understood as beingmuch greater than Δt and/or long compared with known or expectedshort-term field disturbances. By way of example, “approximately Xminutes” could be in a range of 5-10 minutes. Other values could be usedas well.

In still further accordance with example embodiments, while operating inthe background detection state, the HMD will continue to monitor thebackground magnetic field. More particularly, once the baseline valuesof sample mean and standard deviation have been established, the HWM maycontinually update the values to the current time by using ongoingmeasurements of {right arrow over (H)}_(i) from the magnetometer, and byapplying a sliding window as described in equations (2c) and (3c), forexample.

The background magnetic field strength, in turn, provides a baselineagainst which the HWM may at later times be detected as a perturbation.If appropriate conditions are met, as described below, a detectedperturbation may serve as a trigger for the HMD to transition tooperating in the gesture detection state.

The equations given above, as well as additional ones given below, aremathematical formulae for the various defined quantities (e.g., samplemean, standard deviation). As such, the formulae can be applied as givento calculate any of the defined quantities. It will be appreciated,however, that implementations of mathematical formulae in the form ofexecutable instructions on a processor (or the like) for computationalpurposes may employ alternative, analogous algorithmic formulations forreasons of efficiency, speed, or other practical considerations.Accordingly, implementations on an example HMD of computationscorresponding to any formulae and equations described herein should notbe viewed as limited algorithmically only to the exact forms of thoseformulae and/or equations as given.

d. Trigger Determination

FIG. 5 is a conceptual illustration of the perturbation of thebackground magnetic field produced by the HWM. A user 501 is depicted aswearing a HWM 502 that is equipped with a magnetometer device (notshown). By way of example, the user 501 is wearing a HWM in the form ofa magnetic ring 504 on his left hand. Representative field lines of thering 504 of a dipole aligned along the user's finger are shown forillustrative purposes only. Also by way of example, a reference distance505 delineates a spherical volume, represented by circular arc 507,within which a perturbation field caused by the ring 504 makes adetectable contribution to the total field. For purposes ofillustration, the reference distance 505 is depicted as measured fromthe front and center of the HMD 502; in practice a more accurate measuremight be referenced to the location of the magnetometer on the HMD 502.

When the user's hand is located beyond the reference distance 505 at theposition labeled “(1),” only the background field is detected. This isdepicted in the inset 508-1, which represents the magnetic fieldmagnitude ∥H∥ as function of time (solid horizontal line). When theuser's hand is located within the reference distance 505 at the positionlabeled “(2),” the perturbation field makes a detectable contribution tothe total magnetic field magnitude. This is depicted in the inset 508-2,which shows the background magnetic field magnitude ∥H∥ as function oftime, as well as the total of the background magnetic field plus theperturbation field (dashed horizontal line). Note that since the totalof the background plus the perturbation fields is a vector sum, themagnitude of the total field could actually be below that of just thebackground field alone. The depiction in the inset 508-2 of the totalfield magnitude being greater than that of the background isillustrative.

A trigger condition can occur when the magnitude of the total magneticfield (i.e., the vector sum of the background plus the perturbationfields) differs from that of just the background field by at least athreshold amount. In this case, a perturbation of the background fieldis said to be greater than a perturbation threshold, and the referencedistance 505 may accordingly be considered a threshold distance.Determination of a trigger condition may include not only detection of aperturbation above a perturbation threshold, but also when (and howsuddenly) the detected perturbation occurs, and how long it lasts.

In accordance with example embodiments, the HMD may determine theoccurrence of a trigger condition by detecting a perturbation of thebackground magnetic field caused by the HWM. The threshold condition cancorrespond to the HWM being moved within a threshold distance of themagnetometer, where the threshold distance can be considered a distancewithin which hand gestures are expected to occur. Hence, occurrence ofthe trigger condition can signal the start of a gesture, and cantherefore be used to cause the HMD to transition to the gesturedetection state (described below).

More specifically, for each sample time t_(i), a deviation d_(i) ofh_(i) from the sample mean can be determined. By comparing d_(i) to athreshold deviation θ, a “trigger function” may be defined for eachsample time as:

$\begin{matrix}{{Trigger}_{i} = \left\{ {\begin{matrix}{{1\mspace{14mu}{if}\mspace{14mu} d_{i}} > \theta} \\{0\mspace{14mu}{otherwise}}\end{matrix}.} \right.} & (4)\end{matrix}$

The deviation d_(i) can be defined in a number of ways. One example issimply an absolute value of a difference from the sample mean:d _(i) =|h _(i)−μ|.  (5)Note that absolute value is used because any given deviation can beabove or below the sample mean. In this case, d_(i) and θ are in unitsof magnetic field strength (e.g., Telsa or Gauss). As an alternative,the deviation can be defined as a dimensionless number relative to thestandard deviation:

$\begin{matrix}{d_{i} = {\sqrt{\frac{1}{\sigma^{2}}\left( {h_{i} - \mu} \right)^{2}}.}} & (6)\end{matrix}$In this case, d_(i) and θ are in units of standard deviations. Forexample, d_(i)=3.5 corresponds to 3.5 standard deviations (i.e., 3.5 σor “3.5 sigma”). Other forms of deviation d_(i) could be defined aswell, and the example embodiments described herein are not limited onlyto the two above examples.

For either of the two above example definitions of the deviation, thesample mean against which h_(i) is compared could take on one of theforms in equations (2a), (2b), or (2c) above. For example, μ as given byequation (2a) corresponds to an assumed constant (or nearly constant)baseline value. Taking μ as given by equations (2b) or (2c) correspondsto an adaptive value of sample mean, as explained above. For the form ofd_(i) given by equation (6), σ could take on one of the forms inequations (3a), (3b), or (3c) above, in which case the characterizationof a constant (or nearly constant) baseline value of sample mean versusan adaptive value of sample mean applies to the standard deviation aswell.

In further accordance with example embodiments, the form of d_(i) givenby equation (6) will be adopted in evaluating the trigger function ofequation (4). Thus, the perturbation threshold may be written as θ×σ. Assuch, the HMD may determine an occurrence of a trigger by determining atleast one occurrence of d_(j)>θ×σ, or equivalently Trigger_(j)=1, atsome time t_(i=j). That is, for t_(i), i=1, . . . , K, there is at leastone time t_(i=j), 1≦j≦K, for which Trigger_(j)=1. For purposes of thediscussion herein, Trigger_(i)=1 for any i will be referred to as a“trigger event.”

In order to help reduce or eliminate instances of spurious detections ofperturbations above the threshold that can lead to false triggers, thetrigger condition may also include a minimum number of trigger eventswithin times t_(i), i=1, . . . , K. For example, a trigger conditioncould correspond to at least p consecutive occurrences of triggerevents, where 1≦p≦K. Alternatively, a trigger condition could correspondto at least any p out of K occurrences of trigger events, where again1≦p≦K.

In practice, it is convenient to determine if the onset of a triggercondition occurs at the current time t₀ of a sliding time windowW_(0,N). For trigger condition determination based on just a singleoccurrence of a trigger event (i.e., for p=1), this just corresponds toTrigger₀=1. For p>1, the determination corresponds to determining at t₀either p consecutive occurrences of trigger events, or any p out of thelast K occurrences of trigger events. Both approaches may be illustratedby way of example.

Taking p=3 as an example, a trigger condition at time t₀ based on pconsecutive occurrences of trigger events corresponds to Trigger_(j)=1for j=0, 1, 2, where t₁ and t₂ are the next two earlier (consecutive)sample times before t₀. In this case, the onset of a trigger conditionmay be considered as occurring at time t₀ (i.e., the time of the thirdof three consecutive triggers).

Alternatively and again as an example, taking p=3 and K=6 (≦N), atrigger condition at time t₀ based on p out of K occurrences of triggerevents corresponds to Trigger_(j)=1 both for j=0 and for any twoadditional values of j in the range 1≦j≦5, where t₅ are the next fiveearlier (consecutive) sample times before t₀. In this case, the onset ofa trigger condition may again be considered as occurring at time t₀(i.e., the time of the third of three out of six triggers).

In practice, both p and K are kept small since it is generally desirableto make a trigger determination over a small number of samples comparedwith a number of samples in a typical gesture (as discussed below). Inaccordance with example embodiments, trigger determination may beaccomplished with p and K in a range of 1-5% of a typical number ofsamples in a typical gesture. In addition, keeping N>>p and K helpsensure that the sliding time window W_(0,N) spans a time interval largeenough to average out transient fluctuations of the background field,including possible valid triggers. In accordance with exampleembodiments, N≧1,000×p, K, although other relative sizes of N and p, Kmay be used that satisfy N>>p, K.

FIG. 6 is a conceptual illustration of determination of a triggercondition based on a perturbation of the background magnetic field bythe HWM. A user 601 is depicted as wearing a HWM 602 that is equippedwith a magnetometer device (not shown). By way of example, the user 601is wearing a HWM in the form of a magnetic ring 604 on his left hand.Also by way of example, a threshold distance 603 delineates a sphericalgesture detection region, represented by circular arc 605, within whicha perturbation field caused by the ring 604 results in a triggercondition being satisfied. For purposes of illustration, the thresholddistance 603 is depicted as measured from the front and center of theHMD 602; in practice a more accurate measure might be referenced to thelocation of the magnetometer on the HMD 602.

The user's left hand (and ring 604) is shown at three positions, labeled“(1),” “(2),” and “(3),” intended to represent motion of the user's lefthand from position (1) to position (3). At position (1), the ring 604 ispositioned beyond the threshold distance, and moving toward position(2). At position (2), the ring 604 is positioned at (or nearly at) thethreshold distance, and approaching the spherical boundary delineated bythe circular arc 605 as it moves toward position (3). At position (3),the ring 604 is positioned within the threshold distance (and possiblystill moving within the gesture detection region).

As the user's left hand moves from position (1) to position (3), theperturbation of the background magnetic field by the magnetic ring 604may be monitored and measured by the magnetometer of the HMD 602 inorder to determine if and when a trigger condition occurs. This isillustrated in the insets 606-1, 606-2, and 606-3, which depictsuccessive samples of field strength ∥H∥ as the ring 604 moves acrossthe three example positions. Each inset shows a background fieldstrength labeled μ (solid horizontal line), and a perturbation thresholdlabeled +θ and −θ (dashed horizontal lines) above and below thebackground. The successive samples, corresponding to h_(i) in thediscussion above, are shown as diamonds with short vertical lines at therespective sample times.

The inset 606-1 shows the samples h_(i) at sample times t_(i+1), . . . ,t_(i+k), as the ring 604 moves from position (1) toward position (2).The inset 606-2 shows the samples h_(i) at sample times t_(i+k+1),t_(i+k+2), . . . , t_(i+2k), as the ring 604 moves from position (2)towards and across the spherical boundary delineated by the circular arc605. The inset 606-3 shows the samples h_(i) at sample times t_(i+2k+1),t_(i+2k+2), . . . , t_(i+3k), as the ring 604 moves from across thespherical boundary toward position (3).

As illustrated, the samples h_(i) at sample times t_(i), t_(i+1), . . ., t_(i+k) in the inset 606-1 evidently fluctuate near the sample mean μ,but remain within ±θ of the background. Accordingly, a trigger conditionhas not yet occurred. The samples h_(i) at sample times t_(i+k+1),t_(i+k+2), . . . , t_(i+2k), (inset 606-2) show increasing deviationsfrom μ, with the last two, at times t_(2k−1) and t_(i+2k), falling below−θ (that is, more than θ below μ). Referring again top in the abovediscussion, and taking p=2 by way of example, a trigger condition may beconsidered occurring at time t_(i+2k) in the example illustrated FIG. 6.Alternatively, for p=1, a trigger condition may be considered occurringat time t_(i+2k−1.) This illustration also demonstrates that a triggercondition can correspond to a perturbation that is more than a thresholdbelow (i.e. less than) background field.

Continuing with the example illustrated in FIG. 6, the samples h_(i) atsample times t_(i+2k+1), t_(i+2k+2), . . . , t_(i+3k), (inset 606-3)show deviations that are predominantly more than θ below μ. These couldrepresent samples taken while a gesture is being made within the gesturedetection region. Note that a few of the samples in the inset 606-3appear to be within θ of μ. However, once the trigger condition isdetected (e.g. at time t_(i+2k)), it may remain in effect until one ormore other conditions or events occur.

More particularly, the time at which a trigger condition is detected isreferred to as T_(start), and corresponds to the onset of the triggercondition. In accordance with example embodiments, the HMD willtransition to operating in the gesture detection state upondetermination of occurrence of a trigger condition at T_(start). The HMDwill continue to operate in the gesture detection state until one ormore other conditions or combinations of conditions is determined by theHMD to occur, at which time the HMD will transition back to operating inthe background detection state. As discussed in detail below, theseother conditions include cessation of the trigger condition, expirationof a minimum detection window interval W_(min), and detection of agesture. There may be other conditions as well.

e. Gesture Detection and Gesture Detection State

Upon transitioning to operating in the gesture detection state, the HMDwill use successive samples of the vector magnetic field {right arrowover (H)}_(i)=[(H_(x))_(i), (H_(y))_(i), (H_(z))_(i)] at sample timest_(i), i=1, 2, . . . ,N to determine discrete time derivatives of {rightarrow over (H)}_(i) along the {circumflex over (x)}, ŷ, and {circumflexover (z)} axes. More specifically, discrete time derivatives may becomputed as:

$\begin{matrix}{{\left. \begin{matrix}{\left( \frac{\mathbb{d}H_{x}}{\mathbb{d}t} \right)_{i} = {\frac{1}{\Delta\; t}\left\lbrack {\left( H_{x} \right)_{i + 1} - \left( H_{x} \right)_{i}} \right\rbrack}} \\{\left( \frac{\mathbb{d}H_{y}}{\mathbb{d}t} \right)_{i} = {\frac{1}{\Delta\; t}\left\lbrack {\left( H_{y} \right)_{i + 1} - \left( H_{y} \right)_{i}} \right\rbrack}} \\{\left( \frac{\mathbb{d}H_{z}}{\mathbb{d}t} \right)_{i} = {\frac{1}{\Delta\; t}\left\lbrack {\left( H_{z} \right)_{i + 1} - \left( H_{z} \right)_{i}} \right\rbrack}}\end{matrix} \right\}\mspace{14mu}{for}\mspace{14mu} t_{i}},{i = 1},\ldots\mspace{14mu},{N - 1.}} & (7)\end{matrix}$

As defined above, Δt is the sampling interval. Note that there may beone fewer time derivative for each axis than there are samples, sinceeach computed time derivative uses two samples.

The sample times t_(i), i=1, 2, . . . , N for the computed timederivatives are generally different from the similarly-indexed timesdiscussed above in connection with background detection and triggerdetermination. In particular, the samples {right arrow over(H)}_(i)=[(H_(x))_(i), (H_(y))_(i), (H_(z))_(i)] are acquired entirelyor predominantly during a “gesture detection window” W_(g) that begins(opens) at time T_(start) and ends (closes) at T_(stop)>T_(start). Thegesture detection window corresponds to a time interval during which theHWM is presumed to be tracing out a gesture within the gesture detectionregion. Accordingly, the discrete derivatives computed according toequation (7) correspond to derivatives specific to motion of the HWMpresumed to correspond to a particular gesture that is traced during thegesture detection window. As discussed below, the duration of W_(g),given by T_(stop)−T_(start), will generally be at least as long as theminimum detection window interval W_(min), and may be longer dependingon how long the trigger condition persists.

FIG. 7 is a conceptual illustration of samples {right arrow over(H)}_(i)=[(H_(x))_(i), (H_(y))_(i), (H_(z))_(i)] acquired as a HWMtraces out an example gesture while the HMD is in the gesture detectionstate. A user 701 is depicted as wearing a HWM 702 that is equipped witha magnetometer device (not shown). By way of example, the user 701 iswearing a HWM in the form of a magnetic ring 703 on his left hand, whichis depicted as detached from the user's body in order to avoid anoverly-complicated figure. The user's left hand (and ring 703) is shownat four positions, labeled “(1),” “(2),” “(3),” and “(4),” intended torepresent motion of the user's left hand tracing a gesture. At eachposition in this example, the ring 703 is presumed to be within thegesture detection region.

As the ring 703 moves through the gesture (as represented by the fourpositions), the magnetometer acquires samples of {right arrow over(H)}_(i)=[(H_(x))_(i), (H_(y))_(i), (H_(z))_(i)] at each sample time.Sample acquisition is represented in the three insets 706-x, 706-y, and706-z, which respectively show the strength of one of H_(x), H_(y), orH_(z) as a function of time. The successive samples corresponding to(H_(x))_(i), (H_(y))_(i), (H_(z))_(i), are shown as diamonds with shortvertical lines at the respective sample times. The sample times in thisexample, t_(i+k), are the same for each axis. The particular valuesdepicted for (H_(x))_(i), (H_(y))_(i) and (H_(z))_(i) are arbitrary,serving only as conceptual illustration.

In accordance with example embodiments, the HMD will compute thediscrete derivatives according to equation (7) (or an algorithmicallyanalogous form) to obtain a set of computed time derivatives, and thencompare the set of computed time derivatives with one or more storedsets of pre-determined time derivatives in order to find a match. Eachof the one or more stored sets will include a pre-determined associationwith a respective known gesture. Upon finding a match (or a closestmatch, as described below) between the set of computed time derivativesand a particular one of the stored sets of pre-determined timederivatives, the HMD may identify the respective known gestureassociated with the particular stored set as the user's just-madegesture detected via the field {right arrow over (H)}_(i) sampled duringthe gesture detection window.

Each set of computed time derivatives corresponds to a triplet ofcomputed sequences of time derivatives (one sequence each for{circumflex over (x)}, ŷ, and {circumflex over (z)}). Similarly, each ofthe one or more stored sets of pre-determined time derivativescorresponds to a stored triplet of pre-determined sequences of timederivatives (again, one sequence each for {circumflex over (x)}, ŷ, and{circumflex over (z)}). Finding a match (or a closest match) then may beaccomplished by simultaneously matching (or most closely matching) eachrespective computed ({circumflex over (x)}, ŷ, or {circumflex over (z)})sequence of the triplet of computed sequences of time derivatives witheach respective pre-determined ({circumflex over (x)}, ŷ, or {circumflexover (z)}) sequence of one of the stored triplets of pre-determinedsequences of time derivatives.

Because the computed time derivatives are based on differences betweensuccessive samples of {right arrow over (H)}_(i)=[(H_(x))_(i),(H_(y))_(i), (H_(z))_(i)], the background magnetic field contribution to{right arrow over (H)}_(i) is subtracted out. Similarly, each storedtriplet of pre-determined sequences of time derivatives is based ondifferences between a reference set of successive samples of {rightarrow over (H)}_(j), so again, the background magnetic fieldcontribution to {right arrow over (H)}_(j) is subtracted out.Advantageously, any variations in the background magnetic fields betweenthe samples {right arrow over (H)}_(i) and {right arrow over (H)}_(j)may be removed by the subtraction.

As described below, each stored set of pre-determined time derivativescorresponds to a respective known gesture that was previously carriedout during operation of the HMD in a gesture recording state, in whichthe magnetometer acquired samples of {right arrow over (H)}_(i) and theHMD computed and recorded the associated derivatives. As such, eachstored set provides a reference set of time derivatives against which asubsequently detected, presumed gesture may be compared. In practice, asubsequent gesture traced by a HWM and detected by the magnetometer maynot necessarily trace out a path strictly identical to that of apre-determined gesture, even when the subsequent gesture is the same(e.g., intended by a user to be the same) as the pre-determined one.Moreover, the total number of derivatives computed for the subsequentgesture could be different from that of the stored set of pre-determinedderivatives for the same (known) gesture. Accordingly, for any given setof computed time derivatives for a presumed gesture, the HMD may notnecessarily find a strictly exact match with one of the stored sets ofpre-determined time derivatives. As suggested above, finding a “closestmatch” or “most closely matching” may be sufficient to identify a knowngesture with measurements of a presumed gesture.

More particularly, finding a “closest match” or “most closely matching”may be accomplished by treating each stored set of pre-determined timederivatives as a mathematical model of a respective known gesture, andfurther, by statistically evaluating a fit of the computed timederivatives to each model. The HMD could determine the closest matchfrom among the stored sets as the one that yields the best statisticalfit. Note that the HMD could also apply a statistical threshold formatching, such that failure to achieve at least the statisticalthreshold for any of the stored sets (models) in any given instancecould correspond to failure to identify the measured, presumed gesturewith any known gesture. It will be appreciated that there are variousmethods for quantitatively evaluating a fit of measured data to a model,or for quantitatively evaluating a fit of one set of measured data toanother set of measured (or stored) data. It will be further appreciatedthat there other methods that could be used for determining a matchbetween the computed time derivatives and one of the stored sets ofpre-determined time derivatives. As an example, a hidden Markov modelgesture recognition algorithm could be used.

In further accordance with example embodiments, upon identifying a knowngesture based on a match (or closest match) as described above, the HMDmay then invoke or carry out one or more actions associated with theknown gesture. For example, the known gesture may be associated with auser input to the HMD, such as causing the HMD to upload a digital phototo a social networking website or server, or selecting a menu itemdisplayed in the display area of the HMD. It will be appreciated thatany number and variety of actionable or passive user inputs to the HMDcould be associated with known gestures stored on the HMD.

As an alternative or an addition to gesture recognition based oncomparison of computed time derivatives with stored sets ofpre-determined time derivatives, a heuristic method could be used torecognize gestures based on characteristics of just the computed timederivatives. An example of such a heuristic approach may be illustratedby considering recognition of a horizontal “sweeping” or “swipe”gesture. Referring again to FIGS. 1 a and 1 b, and taking the lenselements 110 and 112 to define a plane of the field of view of the HMD,such a sweeping or swipe gesture could be considered as corresponding tomovement of the HWM across the field of view of the HMD, with little orno movement perpendicular to the field-of-view plane.

More particularly, assuming the magnetometer to be fixed to the HMD,with its {circumflex over (x)} and ŷ axes lying in the field-of-viewplane and its {circumflex over (z)} axis perpendicular to thefield-of-view plane, motion purely across the field of view of the HMDmight be recognizable as motion for which all or most of the computed{right arrow over (z)} time derivatives are zero (or largely so). If inaddition, analysis of the determined {right arrow over (x)} and {rightarrow over (y)} time derivatives indicates horizontal motion, ahorizontal sweeping or swipe gesture could then be inferred. Finally,analysis of the signs of the {right arrow over (x)} and {right arrowover (y)} time derivatives could be used to determine the direction(i.e., leftward or rightward) of the horizontal sweeping or swipegesture. Accordingly, such a horizontal sweeping or swipe gesture couldbe recognized without reference to stored sets of pre-determined timederivatives of known gestures. It will be appreciated that such anapproach could be extended to enable recognition of vertical or diagonalsweeping or swipe gestures, as well as line-of-sight gestures, andpossibly other gestures as well.

As noted above, the samples {right arrow over (H)}_(i)=[(H_(x))_(i),(H_(y))_(i), (H_(z))_(i)] used in computing the time derivatives areacquired at sample times t_(i), i=1, 2, . . . , N, entirely orpredominantly within a gesture detection window W_(g) that begins attime T_(start) and ends at T_(stop)>T_(start). In addition, W_(g) is atleast as long as the minimum detection window interval W_(min). Aminimum length time window provides for collection of a minimum numberof samples {right arrow over (H)}_(i)=[(H_(x))_(i), (H_(y))_(i),(H_(z))_(i)] and thereby helps ensure a sufficient number of computedtime derivatives for effectively comparing with the storedpre-determined sets of time derivatives. A minimum length time windowalso can help ensure that occurrences of cessation of the triggercondition do not prematurely end the gesture detection process. Thegesture detection window W_(g) may extend beyond W_(min) if the triggercondition persists beyond W_(min). Hence, the end of W_(g) at T_(stop)corresponds to a time at least W_(min) beyond T_(start) at which the HMDdetermines the trigger condition to cease.

In accordance with example embodiments, upon the start of a gesturedetection window W_(g) at T_(start), the HMD will begin tracking themagnetic field magnitude samples h_(i) for times t_(i)≧T_(start) inorder to monitor for cessation of the trigger condition. TakingT_(cease) to be the time that the HMD determines cessation of thetrigger condition, and Δt_(min) to be the length (duration) of theminimum time window W_(min), the HMD will determine T_(stop) as thelonger of (i.e., the maximum of) T_(cease) and the end of W_(min):T _(stop)=max[T _(cease),(T _(start) +Δt _(min))].  (8)

According to equation (8), if the HMD detects cessation of the triggercondition before the end of W_(min), the gesture detection window willremain open until at least the end of W_(min). If the HMD detectspersistence of the trigger condition beyond the end of W_(min), thegesture detection window will remain open until at least T_(cease). Itis convenient to specify the length (duration) of W_(min) in terms ofnumber of sample intervals, N_(min).

In further accordance with example embodiments, the HMD may determinethat a trigger condition has ceased by determining a threshold number ofconsecutive sample deviations d_(i) below the perturbation threshold, oralternatively, by determining a threshold fractional number of sampledeviations d_(i) measured below the perturbation threshold. Eitheralternative form for T_(cease) can be determined by computing thetrigger function Trigger_(i) over a sliding time window W_(0,K), where Kis small compared with N_(min). This corresponds to searching forcessation over a time window W_(0,K) that is short compared with W_(min)and that slides across W_(g) starting from T_(start). By way of example,K could be approximately 0.06×N_(min), though other values could satisfyK<<N_(min).

Initially, t₀ of the W_(0,K) is T_(start). Thereafter t₀ incrementsacross W_(g) as the window slides with each new sample interval. At anytime t₀ for which the last q≦K consecutive values of Trigger_(i)=0, theHMD may determine that cessation of the trigger condition has occurred.Alternatively, any time t₀ for which the last q≦K out of K values ofTrigger_(i)=0, the HMD may determine that cessation of the triggercondition has occurred. These two alternatives may be respectivelyexpressed analytically as:T _(cease) =t ₀ if Σ_(i=0) ^(q≦k)Trigger_(i)=0,  (9a)andT _(cease) =t ₀ if Σ_(i=0) ^(K−1)Trigger_(i) ≦K−q,q≦K.  (9b)Accordingly, the HMD may use either of equations (9a) or (9b) (or analgorithmically analogous form) to determine T_(cease), and equation (8)to determine T_(stop). Note that the HMD may apply equation (9a) or (9b)repeatedly in instances where either yieldsT_(cease)<T_(start)+Δt_(min).

In accordance with example embodiments, the HMD may accumulate magneticfield samples {right arrow over (H)}_(i) in a buffer (e.g., in memorysuch as program data 208 in FIG. 2) for determination of both h_(i) andthe computed derivatives. While operating in the background detectionstate, the HMD may skip computing discrete derivatives, and only computeh_(i). While operating in the gesture detection state, the HMD maycompute both h_(i) and the discrete derivatives.

In further accordance with example embodiments, the HMD may include inthe computation of the derivatives some samples {right arrow over(H)}_(i) accumulated during just before T_(start.) This could helpimprove the accuracy of gesture detection for instances in which thestart of a gesture precedes determination of an associated triggercondition by a few samples. In practice, the number of pre-triggersamples included in gesture detection (i.e., computed derivatives) wouldbe expected to be small; e.g. approximately 10 or fewer.

In accordance with example embodiments, the HMD may compute the discretederivatives and search for a matching known gesture based on anaccumulation of field samples {right arrow over (H)}_(i) acquired overeither the entire gesture detection window W_(g) or over just a portionof W_(g). More particularly, the HMD may accumulate field samples upuntil T_(stop), and thereafter compute the derivatives and analyze themfor detection of a known gesture. Alternatively, the HMD may begincomputing derivatives and analyzing them for detection of a knowngesture after acquiring a threshold number of samples, where thethreshold number is some fraction of the total number of samplescorresponding to a full gesture detection window. In this alternativeapproach, the HMD may continue to acquire samples as it analyzes them,until a known gesture is detected. This could allow “early” gesturedetection in instances when less than a full number of samples aresufficient to make the determination.

In further accordance with example embodiments, the HMD may transitionback to operating in the background detection state at T_(stop). As analternative, the HMD may transition back to the background detectionstate upon either identification of a known gesture based on analysis ofthe computed derivatives (i.e., comparison with pre-stored derivatives),or failure to identify any known gesture. As still a furtheralternative, the HMD may remain in the gesture detection state if thetrigger condition persists even upon or after identification of a knowngesture. In this case, the HMD may begin a new gesture detection cycleusing new set of acquired samples, without first transitioning back tothe background detection state. For example, a user might make multiplegestures without ever moving the HWM (e.g., magnetic ring) outside ofthe gesture detection region.

f. Gesture Recording and Gesture Recording State

As described above, analysis of computed time derivatives of {rightarrow over (H)}_(i) for gesture recognition may be based on comparingthe computed derivatives with one or more stored sets of pre-determinedtime derivatives, where each stored set is respectively associated witha known gesture. In accordance with example embodiments, each set ofpre-determined time derivatives may be generated and stored on the HMDduring operation in a gesture recording state.

More specifically, during operation in the gesture recording state, theHMD may apply a gesture recording procedure that is largely the same asthe gesture detection procedure described above, except that instead ofmatching computed time derivatives to stored ones, the HMD treats a setof computed time derivatives as a new (or replacement) known gesture andrecords (stores) them together with an identifier of the known gesture.Accordingly, equation (7) again applies to computation of timederivatives. However, while operating in the gesture detection state,the HMD records the computed time derivatives. The HMD also creates athree-way association between the recorded, computed time derivatives,an identifier of a known gesture, and a pre-determined action that maybe carried out by the HMD.

In further accordance with example embodiments, the HMD will transitionto operating in the gesture recording state upon determining theoccurrence of a recording trigger. More particularly, a recordingtrigger may be used to signal to the HMD that the next detected gesture(as traced by the HWM) should be treated as a new or replacementgesture. The recording trigger may also include an indication of anidentifier of the new or replacement gesture, as well as the associatedpre-determined action. Thereafter, when the HWM acquires samples of{right arrow over (H)}_(i) and computes time derivatives from them, itwill store the time derivatives together with the three-way associationfor the gesture. Note that a replacement gesture is one for which thestored time derivatives and three-way association replaces a previousversion of the same for an existing (i.e. previously stored) knowngesture.

In still further accordance with example embodiments, a recordingtrigger may correspond to a particular input or command that includesboth an indication that a new or replacement gesture is about to be made(e.g., the next detected gesture) and the pre-determined action thatshould be associated with the gesture once it is detected and recorded.The trigger could also include the gesture identifier. Alternatively,the gesture identifier could be determined by the HMD as an internalparameter. As still another alternative, a recording trigger couldinclude just an indication to record the time derivatives.Identification of a pre-determined action could be made subsequently.

By way of example, a user could use an already-known gesture as arecording trigger for the next subsequent gesture. Alternatively, a usercould use another form of user input to the HMD to indicate a recordingtrigger. For example, referring again to FIG. 1, a user could use thefinger-operable touchpad 124 to signal a recording trigger. Other formsof user input could be used as well.

In accordance with example embodiments, the HMD could be operating inthe background detection state prior to receiving a recording triggerand responsively transitioning to operating in the gesture detectionstate. Upon completion of recording a new or replacement gesture whileoperating in the gesture recording state, the HMD could then transitionback to operating in the background detection state.

g. Motion-based enhancements and motion compensation

In practical usage of a HMD by a user, there may be instances in whichthe HMD is moving and/or changing orientation relative to the background(geomagnetic) field. For example, the user could be in a moving vehicle,changing direction while walking, or even just shaking his or her headin a way that can alter the alignment between the axes of themagnetometer and the background field. Motion and/or changingorientation can be detected by a HMD using one or more of the motionsensing devices, such as could be part of the motion sensor 232 in FIG.2. For example, an accelerometer, a gyroscope, or a magnetometer, asdescribed below. The effect of motion and/or changing orientation onmagnetometer measurements can manifest in a variety of ways, and cancall for one or another form of compensation applied to operationsdiscussed above, or can provide additional modes of input for enhancinggesture detection based on magnetic sensing.

At a given fixed location, a change in the orientation of themagnetometer (i.e., of the HMD) with respect to the background magneticfield will not change the measured field magnitude at that location.Consequently, such a change in orientation will not generally affect HMDoperations based only on measured field magnitude. For example, a changein orientation that occurs while the HMD is operating in the backgrounddetection state will not affect operation of the magnitude-based triggermechanism discussed above. In contrast, a change in the orientation ofthe magnetometer with respect to the background magnetic field can causea change in the relative strengths of the measured field vectorcomponents H_(x), H_(y), and H_(z). Consequently, such a change inorientation can affect HMD operations based on measured strengths offield vector components. For example, a change in orientation thatoccurs while the HMD is operating in the gesture detection state andover a timescale approximately comparable to that of a typical gesturecan cause the computed time derivatives to deviate from those obtainedin the absence of a change in orientation.

In accordance with example embodiments, while operating in thebackground detection state, the HMD can use a change in the relativestrengths of the measured field vector components H_(x), H_(y), andH_(z) as an alternative trigger. More specifically, even in the absenceof a magnitude-based trigger from a HWM, the HMD may determine that aparticular orientation change detected via a change in the relativestrengths of H_(x), H_(y), and H_(z) corresponds to a pre-determinedtrigger. In response, the HMD may transition to operating in the gesturedetection state. By way of example, the HMD could recognize a particularhead-motion gesture of a user by determining that a change in therelative strengths of measured samples (H_(x))_(i), (H_(y))_(i), and(H_(z))_(i) corresponds to pre-determined, pre-recorded relative changesassociated with the particular head-motion gesture (or trigger). Thepre-determined, pre-recorded relative changes could be established usinga procedure similar to that described above for gesture recording.Further to this example, a user could use a nod or a shake of his or herhead to trigger the start of a hand gesture (i.e., transition of the HMDto the gesture detection state). Other orientation-change-based gesturesand/or triggers could be similarly devised and used as well.

Detected orientation changes could additionally or alternatively signalconditions under which magnetic gesture detection by the HMD could behampered. In further accordance with example embodiments, whileoperating in the background detection state, the HMD can use a change inthe relative strengths of the measured field vector components H_(x),H_(y), and H_(z) to determine that a transition to the gesture detectionstate should be disabled, at least while the orientation is changing.Such an action could be used to disable attempted gesture recognitionunder conditions that might otherwise be prone to an unacceptable orunrecoverable error in computing and/or analyzing time derivatives fromsamples acquired while the orientation is changing. By way of example,the HMD could ignore a magnitude-based trigger from a HWM that occurswhile the orientation is determined to be changing, or at least changingin a manner that does not correspond to a known orientation-change-basedtrigger.

Although a changing orientation can affect computed time derivatives, itmay be possible to compensate for such changing orientation so as topreserve a level of reliability and/or accuracy of gesture detection,even in the presence changing orientation. More specifically, bymeasuring orientation changes concurrently with the acquisition ofsamples of {right arrow over (H)}_(i) during operation in the gesturedetection state, the changing orientation may be accounted for andlargely or entirely removed. The HMD may measure changing orientation bymeans of a motion sensing device, such as the motion sensor 232 in FIG.2. In particular, the sensor may include a gyroscope that can be used bythe HMD to determine orientation changes in a form that can be appliedto analytically adjust the measured samples {right arrow over (H)}_(i)to yield computed time derivatives with motion-related componentslargely or entirely removed.

A change in orientation of the magnetometer with respect to thebackground magnetic field can be represented as a rotation inthree-dimensional space, and expressed analytically in terms of arotation matrix RεSO(3), where SO(3) refers to the “special orthogonalrotation group” in three dimensions. In the absence of rotation, any twosuccessive samples of magnetic field {right arrow over (H)}_(i) and{right arrow over (H)}_(i−1) may be expressed as:{right arrow over (H)} _(i) ={right arrow over (E)}+{right arrow over(M)} _(i),  (10a)and{right arrow over (H)} _(i−1) ={right arrow over (E)}+{right arrow over(M)} _(i−1).  (10b)

In these expressions, {right arrow over (E)} is the Earth's magneticfield (i.e., the background field), assumed to be constant from onesample to the next, and {right arrow over (M)}_(i) and {right arrow over(M)}_(i−1) are the vector contributions from the HWM measured from onesample to the next as the HWM traces out a gesture. Accordingly, thedifference between successive field samples (upon which the computedderivative is based) can be expressed as:Δ{right arrow over (H)} _(i) ={right arrow over (H)} _(i) −{right arrowover (H)} _(i−1) ={right arrow over (M)} _(i) −{right arrow over (M)}_(i−1).  (11)As discussed above, the background field is subtracted out, leaving onlythe difference between the HWM-produced field samples.

When the magnetometer rotates between samples, the difference betweensuccessive field samples can instead be expressed as:(Δ{right arrow over (H)} _(i))_(R) =R _(i) {right arrow over (H)} _(i)−{right arrow over (H)} _(i−1) =R _(i)({right arrow over (E)}+{rightarrow over (M)} _(i))−({right arrow over (E)}+{right arrow over (M)}_(i−1)),  (12)where R_(i) corresponds to the rotation matrix at time t_(i). In thiscase, the background field does not cancel between samples, because ithas been rotated with respect to the magnetometer from one sample to thenext. More particularly, R_(i) specifies the rotation of the magneticfield vector from its direction at time t_(i−1) to its direction at timet_(i). By determining the effect of rotation can be removed.

In accordance with example embodiments, the HMD may be equipped with agyroscope, and may use measurements from the gyroscope to determine R.More specifically, the gyroscope may be configured to measure an angularvelocity three-vector {right arrow over (Ω)} in the same frame ofreference as the magnetometer. The magnitude ω≡∥{right arrow over (Ω)}∥of the angular velocity vector corresponds to an angular rotation rate(e.g., radians per second, or degrees per second), and the direction{right arrow over (ω)}={right arrow over (Ω)}/ω of the angular velocityvector specifies the axis of rotation. For a given increment of time dt,φ=ωdt gives the angle of rotation. The form of the rotation matrix R maybe derived from {right arrow over (Ω)}, ω, {right arrow over (ω)}, andφ, as is known in the art. The effect of R operating on a vector {rightarrow over (v)} is to rotate {right arrow over (v)} around {right arrowover (ω)} by an angle φ in time dt.

By acquiring angular velocity measurements in successive samples {rightarrow over (Ω)}_(i) at discrete sampling times t_(i), i=1, 2, . . . , N,the HMD may compute ω_(i), {circumflex over (ω)}_(i), and φ_(i) todetermine R_(i) at t_(i). The sampling times for the angular velocitymeasurements are taken to the same as those for the magnetic fieldmeasurements, so that Δt=t_(i+1)−t_(i) is the same for all i. The effectof R_(i) operating on {right arrow over (H)}_(i) is to rotate {rightarrow over (H)}_(i) around {circumflex over (ω)}_(i) by an angle φ_(i)in time Δt.

In practice, the rotation represented by R_(i) is already in themeasurement of the magnetic field sample R_(i){right arrow over (H)}_(i)in equation (12). However, the effect of the rotation on the measurementcan be removed by using the inverse of the rotation matrix.Specifically, let the inverse of R_(i) be R_(i) ⁻¹. Then applying R_(i)⁻¹ to the measurement R_(i){right arrow over (H)}_(i) in equation (12)gives:R _(i) ⁻¹ R _(i) {right arrow over (H)} _(i) −{right arrow over (H)}_(i−1) =R _(i) ⁻¹ R _(i)({right arrow over (E)}+{right arrow over (M)}_(i))−({right arrow over (E)}+{right arrow over (M)} _(i−1))={rightarrow over (M)} _(i) −{right arrow over (M)} _(i−1).  (13)Thus, the Earth's magnetic field again cancels, and the un-rotated formof Δ{right arrow over (H)}_(i) in equation (11) may be recovered. Itwill be appreciated that R_(i) ⁻¹ can be determined from R_(i) by knownmethods.

In accordance with example embodiments, the HMD may acquire samples{right arrow over (Ω)}_(i) and determine R_(i) and R_(i) ⁻¹ at sampletimes t_(i). The HMD may then use equation (13) (or an algorithmicallyanalogous form) to compute discrete time derivatives corrected forchanges in orientation of the magnetometer with respect to thebackground magnetic field occurring (or that occurred) while magneticfield samples {right arrow over (H)}_(i) are being (or were) acquired,also at sample times t_(i). Advantageously, the gesture recognitionbased on magnetic sensing of a HWM may thus be carried out while themagnetometer of the HMD is undergoing a change in orientation withrespect to the background magnetic field.

h. Multiple Magnetometers

In accordance with example embodiments, a magnetometer device of a HMDmay include more than one three-axis magnetometer. In particular, amagnetometer device may include two three-axis magnetometers mounted onthe HMD with a baseline distance between them. For example, they couldbe mounted at opposite ends of eyeglasses 102 in FIGS. 1 a and/or 2 a.In such a configuration, the two magnetometers could provide the HMDwith a stereoscopic view of time derivatives associated with HWM-basegestures.

More specifically, for identical magnetometers “Mag1” and “Mag2” mountedon an HMD and separated by the typically small width of eyeglasses(e.g., approximately 4-5 inches), the background magnetic field (e.g.,the geomagnetic field) will be identical for practical purposesconsidered herein. If the axes of Mag1 and Mag2 are identicallyoriented, then the vector components of the background field will alsobe identical for like axes of Mag1 and Mag2. Accordingly, Mag1 and Mag2can be calibrated in the absence of any disturbance to the backgroundfield to ensure that they both measure the same vector component valuesof the background magnetic field.

In particular, the background magnetic field measured at any givensample time t_(i), i=1, 2, . . . , N, by Mag1 and Mag2 will be the same.Thus, the magnetic fields measured respectively by Mag1 and Mag2 at timet_(i) can be expressed as {right arrow over ((H₁))}_(i) and {right arrowover ((H₂))}_(i), where {right arrow over ((H₁))}={right arrow over((H₂))}_(i). It also follows that (h_(i))_(i)=(h₂)_(i) where (h₁)_(i)and (h₂)_(i) are correspond h_(i) determined from {right arrow over((H₁))}_(i) and {right arrow over ((H₂))}_(i), respectively. As aconsequence, subtracting the background field magnitude measured by Mag1from that measured by Mag2 on a sample-by-sample basis can zero out thebackground, since (h₁)_(i)−(h₂)_(i)=0.

Since the size scale of the gesture detection region is typically onlyslightly larger than, but of the same order of magnitude of, theseparation between Mag1 and Mag2, the respective angular displacementsbetween the HWM and each of Mag1 and Mag2 will generally be measurablydifferent during gesture detection (i.e., while the HWM is within thegesture detection region). At the same time, because Mag1 and Mag2measure the same background field, the perturbation-based triggeringmethod described above may be applied with greater sensitivity, sincethe background field contribution can be removed prior to monitoring forperturbations. Thus, the use of Mag1 and Mag2 to may allow for moresensitive triggering of gesture detection.

In addition to using Mag1 and Mag2 to cancel out the background magneticfield, each could also function individually in the manner describedabove for just a single magnetometer. The HMD could then combine themeasurements of to the two magnetometers to support a more precisetrigger definition, enhance the accuracy of trigger detection anddetermination while operating in the background detection state, enhancethe accuracy of gesture recognition while operating in the gesturedetection state, and support the creation and recognition of morecomplex gestures.

In further accordance with example embodiments, while operating in thebackground detection state, the HMD may determine the occurrence of atrigger condition by determining that measurements from bothmagnetometers concurrently indicate a magnitude-based trigger conditionfrom the HWM. Alternatively, the HMD may determine the occurrence of atrigger condition by determining that measurements from at least one ofthe magnetometers indicate a magnitude-based trigger condition from theHWM. Similarly, orientation-change-based trigger detection could bedetermined from concurrent orientation-change-based triggers from bothmagnetometers, or an orientation-change-based trigger from at least oneof the magnetometers.

In still further accordance with example embodiments, gesturerecognition using two magnetometers could combine individual timederivatives from both magnetometers into a three-dimensional (e.g.,stereoscopic) view of time derivatives. This could introduce one or moreadditional spatial dimensions to gesture creation and gesturerecognition. As an example, a hand gesture in the form a circle might beused to enclose an area of display viewed within the field of view ofthe HMD. The HMD might then create a digital photograph of the enclosedarea. By using two magnetometers to recognize the circular gesture, thedistance of the HWM from the HMD when the gesture is made could bedetermined, and the size of the enclosed area in the field of viewthereby adjusted according to the determined distance. In this way, asingle gesture could produce different results when analyzed by the HMD,based (in this example) on distance from the HMD. Other modes ofcombining derivatives from two magnetometers to enhance gesturerecognition are possible as well.

i. Higher-Order Multipole Magnetic Elements and Alternate Forms of HWMs

In accordance with example embodiments, the HWM may include a magneticelement that produces a magnetic field of higher order than just adipole field. For example, the HWM could produce a quadrupole magneticfield. As is known, the magnitude of a quadrupole magnetic field fallsoff as r⁻⁴, where r is the distance from the source of the field.Consequently the gesture detection region for a HWM with a quadrupolefield could be smaller and have a sharper boundary than one with just adipole magnetic field, since a dipole magnetic field falls off as asr⁻³. Moreover, a HWM could include multiple dipole magnetic elements, orcombined dipole and quadurpole magnetic elements, configured in relativeorientations so as to produce field geometries that are non-spherical atdistances within a gesture detection region. This could be used tocreate a non-spherical gesture detection region.

In further accordance with example embodiments, the HWM could take aform other than a single ring. For example, the HWM could be in form ofmultiple rings, or other hand-worn jewelry. The HWM could also be in theform of a magnetic decal affixed to one or more fingernails, or one ormore magnetize artificial fingernails. Combining a multiple-finger HWMwith one or more three-axis magnetometers could allow recognition ofcomplex gestures involving combined motions of two or more fingers, inaddition to bulk hand motion. Moreover, the HWM could take the form of afashionable or stylish adornment having potential marketing value beyondits function in gesture sensing by the magnetometer device.

j. Example Method

The example embodiments for magnetometer-based gesture detection andrecognition described above in operational terms of can be implementedas a method on a wearable HMD equipped with a magnetometer device. Anexample embodiment of such a method is described below.

FIG. 8 is a flowchart illustrating an example embodiment of a method ina wearable computing system, such as a wearable HMD, formagnetometer-base gesture detection and recognition. The illustratedsteps of the flowchart could be implemented in the wearable head-mounteddisplay as executable instructions stored in one or another form ofmemory, and executed by one or more processors of the wearablehead-mounted display. The HMD could include a magnetometer ormagnetometer device for carrying out measurements used in the method.Examples of a wearable HMD include the wearable computing system 100 inFIG. 1 and the wearable computing system 202 in FIG. 2. The executableinstructions could also be stored on some form of non-transitorytangible computer readable storage medium, such as magnetic or opticaldisk, or the like, and provided for transfer to the wearablehead-mounted display's memory during configuration or other procedure(s)for preparing the wearable head-mounted display for operation.

As shown, at step 802 the HMD operates in a background detection state.While operating in the background detection state, the HMD carriesvarious functions of the background detection state. These includemeasuring the magnitude of the background magnetic field with amagnetometer device, monitoring for occurrence of a trigger from a HWM,and upon occurrence of the trigger, opening a gesture detection windowand transitioning to operating in a gesture detection state.

At step 804, the HMD operates in the gesture detection state. Whileoperating in the gesture detection state, the HMD carries variousfunctions of the gesture detection state. These include tracking motionof the HWM by measuring three orthogonal components of the magneticfield H_(x), H_(y), and H_(z) with the magnetometer device, determiningtime derivatives of H_(x), H_(y), and H_(z), matching the determinedtime derivatives to known time derivatives of a known gesture,identifying the known gesture, and upon both cessation of the triggerand expiration of a minimum detection window, transitioning to operatingin the background detections state.

In accordance with the example embodiment, measuring the magnitude ofthe background magnetic field with a magnetometer device while operatingin the background detection state (step 802) could be achieved asdescribed in the discussion above of the background detection state.More particularly, the HMD could determine the magnitude of thebackground magnetic field from time sample measurements of threeorthogonal components of the magnetic field H_(x), H_(y), and H_(z) fromthe magnetometer device, as described above in connection with thediscussion of equations (2a), (2b), and (2c).

In further accordance with the example embodiment, the HMD could monitorfor a trigger from the HWM while operating in the background detectionstate (step 802), monitoring for occurrence of a trigger from a HWM bydetermining a the perturbation by the HWM of the background fieldmagnitude remains at least as large as the perturbation threshold for atleast a first threshold length of time. For example, the HMD coulddetermine that a time-average value of the perturbation of thedetermined field magnitude remains at least as large as the perturbationthreshold for at least the first threshold length of time.

More particularly, the HMD could apply the operations described inconnection with equations (4)-(6) in the discussion above of triggerdetermination in order to monitor for occurrence of a trigger. Forexample, the HMD could detect at least a threshold number of deviationsexceeding a threshold perturbation. The time at which the HMD determinesthe trigger to occur could then be taken to be a start time T_(start).The HMD could thus open the gesture detection window and transition tooperating in the gesture detection state at T_(start).

In accordance with the example embodiment, while operating in thegesture detection state (step 804), the HMD could apply the operationsdescribed in connection with equation (7) in the discussion above of thegesture detection state in order to determine the time derivatives ofthe three orthogonal components of the magnetic field H_(x), H_(y), andH_(z) measured at sample times by the magnetometer device.

In further accordance with the example embodiment, while operating inthe gesture detection state (step 804), the HMD could then compare thedetermined time derivatives to known time derivatives of a knowngesture, and thereby identify the known gesture. Details are againdescribed in the discussion above of the gesture detection state.

In further accordance with the example embodiment, specifying bothcessation of the trigger and expiration of a minimum window as acondition for the HMD to transition from the operating in the backgrounddetection state (step 804) provides for a minimum number of magneticfield samples to be acquired and analyzed. Cessation of the triggercould be based on determining that the perturbation by the HWM of thebackground field magnitude remains less than the perturbation thresholdfor at least a second threshold length of time. For example, the HMDcould determine that a time-average value of the perturbation of thedetermined field magnitude remains less than the perturbation thresholdfor at least the second threshold length of time. Once more, details aredescribed in the discussion above of the gesture detection state.

In further accordance with the example embodiment, in responsive toidentifying the respective known gesture, the HMD could identify apre-determined computer-executable action associated with the identifiedknown gesture. The HMD could then carry out the action. In this way,gesture recognition based on magnetic sensing of a HWM can provide ameans of user input and interaction with the HMD.

In further accordance with the example embodiment, the example methodcould include a gesture recording state, as described above. Inaddition, the example method could include motion enhancements and/ormotion compensation, also described above. In still further accordancewith the example embodiment, and once more as described above, theexample method could include additional steps for acquiring andanalyzing magnetic field measurements from a second magnetometer.

It will be appreciated that the steps shown in FIG. 8 are meant toillustrate operation of an example embodiment. As such, various stepscould be altered or modified, the ordering of certain steps could bechanged, and additional steps could be added, while still achieving theoverall desired operation.

FIG. 9 a depicts a state diagram including a background detection stateand gesture detection state, according to an example embodiment.Operation of the HMD in the background detection state 902 includes thefunctions of this state described above. Upon determination that atrigger has occurred, the HMD makes a transition 903 to the gesturedetection state 904. Operation of the HMD in the gesture detection state904 includes the functions of this state described above. Upon bothcessation of the trigger and expiration of a minimum detection window,the HMD make a transition 905 to the background detection state.

FIG. 9 b depicts a state diagram including a background detection stateand gesture recording state, according to an example embodiment.Operation of the HMD in the background detection state 902 againincludes the functions of this state described above. Upon determinationthat a recording trigger has occurred, the HMD make a transition 907 tothe gesture recording state 906. Operation of the HMD in the gesturedetection state 906 includes the functions of this state describedabove. Upon both completion of storing a new or replacement gesture,together with a gesture identifier and the associated recorded timederivatives, the HMD make a transition 909 to the background detectionstate.

It will be appreciated that each of the background detection state, thegesture detection state, and the gesture recording state couldrespectively include other functions not explicitly described herein, solong as any such other functions of that respective state are notexclusive to or contradictory to the functions of that respective statethat are described herein.

CONCLUSION

An illustrative embodiment has been described by way of example herein.Those skilled in the art will understand, however, that changes andmodifications may be made to this embodiment without departing from thetrue scope and spirit of the elements, products, and methods to whichthe embodiment is directed, which is defined by the claims.

What is claimed:
 1. In a wearable head-mounted display (HMD) having aprocessor and a magnetometer device with three orthogonal measurementaxes, a computer-implemented method comprising: operating in abackground detection state; while operating in the background detectionstate, carrying out functions of the background state including,measuring three orthogonal components of a background magnetic fieldwith the magnetometer device, and determining a field magnitude of thebackground magnetic field from the three measured orthogonal components,determining an occurrence of a trigger from a hand-wearable magnet (HWM)at a time T_(start) upon detecting a perturbation by the HWM of thedetermined field magnitude at least as large as a perturbationthreshold, and upon determining the occurrence of the trigger,transitioning to operating in a gesture detection state; and whileoperating in the gesture detection state, carrying out functions of thegesture detection state including, for the duration of the gesturedetection state, tracking motion of the HWM by determining timederivatives of magnetic field strength measured by the magnetometerdevice along each of the three orthogonal measurement axes, making acomparison of the determined time derivatives with one or more sets ofpre-determined time derivatives of magnetic field strength, each of theone or more sets being stored at the wearable HMD and each beingassociated with a respective known gesture, upon matching the determinedtime derivatives with a particular set of the one or more sets based onthe comparison, identifying the respective known gesture associated withthe particular set, and transitioning to operating in the backgrounddetection state upon both of, measuring the magnitude perturbation bythe HWM of the determined field magnitude to be less than theperturbation threshold, and determining an expiration of a time intervalW that begins at T_(start).
 2. The method of claim 1, furthercomprising: responsive to identifying the respective known gestureassociated with the particular set, identifying a pre-determinedcomputer-executable action associated with the identified respectiveknown gesture; and carrying out the identified pre-determinedcomputer-executable action with the processor.
 3. The method of claim 1,wherein detecting the perturbation by the HWM of the determined fieldmagnitude at least as large as the perturbation threshold comprisesdetermining that the perturbation by the HWM of the determined fieldmagnitude remains at least as large as the perturbation threshold for atleast a threshold length of time.
 4. The method of claim 3, whereindetermining that the perturbation by the HWM of the determined fieldmagnitude remains at least as large as the perturbation threshold for atleast the threshold length of time comprises determining that atime-average value of the perturbation of the determined field magnituderemains at least as large as the perturbation threshold for at least thethreshold length of time.
 5. The method of claim 1, wherein measuringthe magnitude perturbation by the HWM of the determined field magnitudeto be less than the perturbation threshold comprises determining thatthe perturbation by the HWM of the determined field magnitude remainsless than the perturbation threshold for at least a threshold length oftime.
 6. The method of claim 5, wherein determining that theperturbation by the HWM of the determined field magnitude remains lessthan the perturbation threshold for at least the threshold length oftime comprises determining that a time-average value of the perturbationof the determined field magnitude remains less than the perturbationthreshold for at least the threshold length of time.
 7. The method ofclaim 1, wherein measuring the three orthogonal components of thebackground magnetic field with the magnetometer device comprisesmeasuring magnetic field components H_(x), H_(y), and H_(z) of amagnetic field vector {right arrow over (H)}=[H_(x), H_(y), H_(z)] atconsecutive discrete times t_(i), i=N₁, . . . , N₂, over a sliding timewindow W_(N) ₁ _(,N) ₂ from N₁ to N₂, wherein N₂=N₁+N−1, and N≧1, andwherein determining the field magnitude of the background magnetic fieldfrom the three measured orthogonal components comprises: computingmagnetic field magnitude samples h_(i)=∥H∥=√{square root over (H_(x)²+H_(y) ²+H_(z) ²)} at the consecutive discrete times t_(i), i=N₁, . . ., N₂ over W_(N) ₁ _(,N) ₂ ; and computing a sample mean field strengthμ_(N) ₁ _(,N) ₂ =1/NΣ_(i=N) ₁ ^(N) ² h_(i) over W_(N) ₁ _(,N) ₂ .
 8. Themethod of claim 7, wherein detecting the perturbation by the HWM of thedetermined field magnitude at least as large as the perturbationthreshold comprises: making a determination that at least one of thefield magnitude samples h_(j) deviates from μ_(N) ₁ _(,N) ₂ by an amountat least as large as the perturbation threshold, wherein N₁≦j≦N₂.
 9. Themethod of claim 8, wherein N≧2, and wherein making the determinationthat the at least one of the field magnitude samples h_(j) deviates fromμ_(N) ₁ _(,N) ₂ by an amount at least as large as the perturbationthreshold comprises: computing a standard deviation$\sigma_{N_{1},N_{2}} = \sqrt{\frac{1}{N - 1}{\sum\limits_{i = N_{1}}^{N_{2}}\left( {h_{i} - \mu_{N_{1},N_{2}}} \right)^{2}}}$ over W_(N) ₁ _(,N) ₂ ; and determining that for the at least one of thefield magnitude samples h_(j), a deviation$d_{j} = \sqrt{\frac{\left( {h_{j} - \mu_{N_{1},N_{2}}} \right)^{2}}{\sigma_{N_{1},N_{2}}^{2}}}$ is at least as large as a threshold deviation θ, wherein theperturbation threshold is θ×φ_(N) ₁ _(,N) ₂ .
 10. The method of claim 9,wherein determining that d_(j) is at least as large as θ for the atleast one of the field magnitude samples h_(j) comprises: determiningthat d_(j)≧θ for at least q values of j over W_(N) ₁ _(,N) ₂ as W_(N) ₁_(,N) ₂ slides across p time increments, wherein q≦p.
 11. The method ofclaim 8, wherein the wearable HMD further includes a motion detector,and wherein making the determination that the at least one of the fieldmagnitude samples h_(j) deviates from μ_(N) ₁ _(,N) ₂ by an amount atleast as large as the perturbation threshold comprises both of:determining that the at least one of the field magnitude samples h_(j)deviates from μ_(N) ₁ _(,N) ₂ by an amount at least as large as theperturbation threshold; and concurrently determining that the motiondetector is detecting motion at no greater than a threshold level ofmotion.
 12. The method of claim 1, wherein determining the timederivatives of magnetic field strength measured by the magnetometerdevice along each of the three orthogonal measurement axes comprises:during at least a portion of the time interval W, measuring orthogonalmagnetic field components H_(x), H_(y), and H_(z) of a magnetic fieldvector {right arrow over (H)}=[H_(x), H_(y), H_(z)] at consecutivediscrete times t_(i), i=1, . . . , N, wherein N≧2, to obtain samples ofthe of H_(x), H_(y), and H_(z) at the consecutive discrete times t_(i),i=1, . . . , N; and computing discrete time derivatives of the measuredorthogonal magnetic field components H_(x), H_(y), and H_(z) fromdiscrete differences between successive samples.
 13. The method of claim12, wherein each of the one or more sets of pre-determined timederivatives of magnetic field strength comprises a respective triplet ofpre-determined sequences of discrete time derivatives of a magneticfield along the three orthogonal measurement axes, and wherein matchingthe determined time derivatives with the particular set of the one ormore sets based on the comparison comprises: determining a closest matchbetween the computed discrete time derivatives and the respectivetriplet of pre-determined sequences of discrete time derivatives of oneof the one or more sets.
 14. The method of claim 12, wherein t_(i) fori=1 is one of t₁<T_(start), t₁=T_(start), and t_(i)>T_(start).
 15. Themethod of claim 1, wherein each of the one or more sets ofpre-determined time derivatives of magnetic field strength comprises arespective triplet of pre-determined sequences of discrete timederivatives of a magnetic field along the three orthogonal measurementaxes, and wherein the method further comprises: receiving a recordingcommand, and responsively transitioning to operating in a gesturerecording state; and while operating in the gesture recording state,carrying out functions of the gesture recording state including,determining an occurrence of a recording trigger from the HWM bydetecting a perturbation by the HWM of the determined field magnitude atleast as large as a recording perturbation threshold, upon determiningthe occurrence of the recording trigger, measuring orthogonal magneticfield components H_(x), H_(y), and H_(z) of a magnetic field vector{right arrow over (H)}=[H_(x), H_(y), H_(z)] at consecutive discretetimes t_(i), i=1, . . . , N, wherein N≧2, to obtain samples of the ofH_(x), H_(y), and H_(z) at the consecutive discrete times t_(i), i=1, .. . , N, computing discrete time derivatives of the measured orthogonalmagnetic field components H_(x), H_(y), and H_(z) from discretedifferences between successive samples to obtain a recorded triplet ofsequences of discrete time derivatives$\left( \frac{\mathbb{d}H_{x}}{\mathbb{d}t} \right)_{i},\left( \frac{\mathbb{d}H_{y}}{\mathbb{d}t} \right)_{i},{{and}\mspace{14mu}\left( \frac{\mathbb{d}H_{z}}{\mathbb{d}t} \right)_{i}},$for N−1 consecutive values of i between i=1 and i=N, creating anassociation between the recorded triplet of sequences of discrete timederivatives, an identifier of a respective pre-determined gesture, andan identifier of a pre-determined computer-executable action, andstoring the recorded triplet of sequences of discrete time derivatives,the identifier of the respective pre-determined gesture, the identifierof the pre-determined computer-executable action, and the createdassociation.
 16. The method of claim 1, further comprising: whileoperating in the gesture detection state, carrying out further functionsof the gesture detection state including, upon matching the determinedtime derivatives with the particular set of the one or more sets basedand identifying the respective known gesture associated with theparticular set, transitioning to operating in the background detectionstate, and upon failing to match the determined time derivatives withthe set of any of the one or more sets, transitioning to operating inthe background detection state.
 17. The method of claim 1, furthercomprising: while operating in the gesture detection state, carrying outfurther functions of the gesture detection state including, uponmatching the determined time derivatives with the particular set of theone or more sets based and identifying the respective known gestureassociated with the particular set: continuing to track motion of theHWM by determining subsequent time derivatives of magnetic fieldstrength measured by the magnetometer device along each of the threeorthogonal measurement axes, making an additional comparison of thedetermined subsequent time derivatives with the one or more sets ofpre-determined time derivatives of magnetic field strength, and uponmatching the determined subsequent time derivatives with a given set ofthe one or more sets based on the additional comparison, identifying therespective known gesture associated with the given set.
 18. The methodof claim 1, further comprising: while operating in the backgrounddetection state, carrying out further functions of the backgrounddetection state including, determining the occurrence of the trigger bydetecting a directional perturbation of the determined field magnitudecorresponding to motion of the wearable HMD with respect to thebackground magnetic field, wherein the directional perturbation is atleast as large as a directional perturbation threshold.
 19. The methodof claim 1, wherein the three orthogonal measurement axes comprise afirst triplet of orthogonal measurement axes of the magnetometer device,wherein the magnetometer device comprises a second triplet of orthogonalmeasurement axes, and wherein the method further comprises: whileoperating in the background detection state, carrying out furtherfunctions of the background state including, measuring three orthogonalcomponents of the background magnetic field with the second triplet oforthogonal axes to determine a supplementary field magnitude of thebackground magnetic field, determining an occurrence of a supplementarytrigger from the HWM by detecting a supplementary perturbation by theHWM of the determined supplementary field magnitude at least as large asa supplementary perturbation threshold, and upon determining theoccurrence of at least one of the trigger and the supplementary trigger,transitioning to operating in the gesture detection state; and whileoperating in the gesture detection state, carrying out further functionsof the gesture detection state including, tracking motion of the HWM bydetermining supplementary time derivatives of magnetic field strengthmeasured with the second triplet of orthogonal axes, making a jointcomparison of the determined time derivatives and the determinedsupplementary time derivatives with one or more combined sets ofpre-determined time derivatives and pre-determined supplementary timederivatives, each of the one or more combined sets being stored at thewearable HMD and each being associated with one of the respective knowngestures, and upon jointly matching both the determined time derivativesand the determined supplementary time derivatives with a particularcombined set of the one or more combined sets based on the jointcomparison, identifying the respective known gesture associated with theparticular combined set.
 20. The method of claim 1, wherein the HWMcomprises a magnet with at least one of: a dipole magnetic field and aquadrupole magnetic field.
 21. In a wearable head-mounted display (HMD)having a processor and a magnetometer device with three orthogonalmeasurement axes, a computer-implemented method comprising: operating ina background detection state; while operating in the backgrounddetection state, carrying out functions of the background stateincluding, measuring three orthogonal components of a backgroundmagnetic field with the magnetometer device, and determining a fieldmagnitude of the background magnetic field from the three measuredorthogonal components, determining an occurrence of a trigger from ahand-wearable magnet (HWM) at a time T_(start) upon detecting aperturbation by the HWM of the determined field magnitude at least aslarge as a perturbation threshold, and upon determining the occurrenceof the trigger, transitioning to operating in a gesture detection state;and while operating in the gesture detection state, carrying outfunctions of the gesture detection state including, for the duration ofthe gesture detection state, tracking motion of the HWM by determiningtime derivatives of magnetic field strength measured by the magnetometerdevice along each of the three orthogonal measurement axes, analyzingthe determined time derivatives to determine a known gesture,identifying a pre-determined computer-executable action associated withthe determined known gesture, and transitioning to operating in thebackground detection state upon both of, measuring the magnitudeperturbation by the HWM of the determined field magnitude to be lessthan the perturbation threshold, and determining an expiration of a timeinterval W that begins at T_(start).
 22. A wearable head-mount display(HMD) comprising: a processor; memory accessible to the processor; amagnetometer device with three orthogonal measurement axes; andexecutable instructions stored in the memory that upon execution by theprocessor cause the HMD to carry out operations including: operating ina background detection state, while operating in the backgrounddetection state, carrying out functions of the background stateincluding: measuring three orthogonal components of a backgroundmagnetic field with the magnetometer device, and determining a fieldmagnitude of the background magnetic field from the three measuredorthogonal components, determining an occurrence of a trigger from ahand-wearable magnet (HWM) at a time T_(start) by detecting aperturbation upon the HWM of the determined field magnitude at least aslarge as a perturbation threshold, and upon determining the occurrenceof the trigger, transitioning to operating in a gesture detection state,and while operating in the gesture detection state, carrying outfunctions of the gesture detection state including: for the duration ofthe gesture detection state, tracking motion of the HWM by determiningtime derivatives of magnetic field strength measured by the magnetometerdevice along each of the three orthogonal measurement axes, making acomparison of the determined time derivatives with one or more sets ofpre-determined time derivatives of magnetic field strength, wherein eachof the one or more sets is stored at the wearable HMD and each isassociated with a respective known gesture, upon matching the determinedtime derivatives with a particular set of the one or more sets based onthe comparison, identifying the respective known gesture associated withthe particular set, and transitioning to operating in the backgrounddetection state upon both of, measuring the magnitude perturbation bythe HWM of the determined field magnitude to be less than theperturbation threshold, and determining an expiration of a time intervalW that begins at T_(start).
 23. The wearable HMD of claim 22, whereinthe operations further include: responding to identifying the respectiveknown gesture associated with the particular set by identifying apre-determined computer-executable action associated with the identifiedrespective known gesture; and carrying out the identified pre-determinedcomputer-executable action with the processor.
 24. The wearable HMD ofclaim 22, wherein measuring the three orthogonal components of abackground magnetic field with the magnetometer device comprisesmeasuring magnetic field components H_(x), H_(y), and H_(z) of amagnetic field vector {right arrow over (H)}=[H_(x), H_(y), H_(z)] atconsecutive discrete times t_(i), i=1, . . . , N₂, over a sliding timewindow W_(N) ₁ _(,N) ₂ from N₁ to N₂, wherein N₂=N₁+N−1 and N≧1, whereindetermining the field magnitude of the background magnetic field fromthe three measured orthogonal components comprises: computing magneticfield magnitude samples h_(i)=∥H∥=√{square root over (H_(x) ²+H_(y)²+H_(z) ²)} at the consecutive discrete times t_(i), i=1, . . . , N₂over W_(N) ₁ _(,N) ₂ ; and computing a sample mean field strength μ_(N)₁ _(,N) ₂ =1/NΣ_(i=N) ₁ ^(N) ² h_(i) over W_(N) ₁ _(,N) ₂ , and whereindetecting the perturbation by the HWM of the determined field magnitudeat least as large as the perturbation threshold comprises making adetermination that at least one of the field magnitude samples h_(j)deviates from μ_(N) ₁ _(,N) ₂ by an amount at least as large as theperturbation threshold, wherein N₁≦j≦N₂.
 25. The wearable HMD of claim24, wherein the wearable HMD further includes a motion detector, whereinmaking the determination that the at least one of the field magnitudesamples h_(j) deviates from μ_(N) ₁ _(,N) ₂ by an amount at least aslarge as the perturbation threshold comprises both of: determining thatthe at least one of the field magnitude samples h_(j) deviates fromμ_(N) ₁ _(,N) ₂ by an amount at least as large as the perturbationthreshold; and concurrently determining that the motion detector isdetecting motion at no greater than a threshold level of motion.
 26. Thewearable HMD of claim 22, wherein each of the one or more sets ofpre-determined time derivatives of magnetic field strength comprises arespective triplet of pre-determined sequences of discrete timederivatives of a magnetic field along the three orthogonal measurementaxes, wherein determining the time derivatives of magnetic fieldstrength measured by the magnetometer device along each of the threeorthogonal measurement axes comprises: during at least a portion of thetime interval W, measuring orthogonal magnetic field components H_(x),H_(y), and H_(z) of a magnetic field vector {right arrow over(H)}=[H_(x), H_(y), H_(z)] at consecutive discrete times t_(i), i=1, . .. , N, wherein N≧2, to obtain samples of the of H_(x), H_(y), and H_(z)at the consecutive discrete times t_(i), i=1, . . . , N; and computingdiscrete time derivatives of the measured orthogonal magnetic fieldcomponents H_(x), H_(y), and H_(z) from discrete differences betweensuccessive samples, and wherein matching the determined time derivativeswith the particular set of the one or more sets based on the comparisoncomprises determining a closest match between the computed discrete timederivatives and the respective triplet of pre-determined sequences ofdiscrete time derivatives of one of the one or more sets.
 27. Thewearable HMD of claim 22, wherein each of the one or more sets ofpre-determined time derivatives of magnetic field strength comprises arespective triplet of pre-determined sequences of discrete timederivatives of a magnetic field along the three orthogonal measurementaxes, and wherein the operations further include: transitioning tooperating in a gesture recording state in response to receiving arecording command; and while operating in the gesture recording state,carrying out functions of the gesture recording state including,determining an occurrence of a recording trigger from the HWM bydetecting a perturbation by the HWM of the determined field magnitude atleast as large as a recording perturbation threshold, upon determiningthe occurrence of the recording trigger, measuring orthogonal magneticfield components H_(x), H_(y), and H_(z) of a magnetic field vector{right arrow over (H)}=[H_(x), H_(y), H_(z)] at consecutive discretetimes t_(i), i=1, . . . , N, wherein N≧2, to obtain samples of the ofH_(x), H_(y), and H_(z) at the consecutive discrete times t_(i), i=1, .. . , N, computing discrete time derivatives of the measured orthogonalmagnetic field components H_(x), H_(y), and H_(z) from discretedifferences between successive samples to obtain a recorded triplet ofsequences of discrete time derivatives$\left( \frac{\mathbb{d}H_{x}}{\mathbb{d}t} \right)_{i},\left( \frac{\mathbb{d}H_{y}}{\mathbb{d}t} \right)_{i},{{and}\mspace{14mu}\left( \frac{\mathbb{d}H_{z}}{\mathbb{d}t} \right)_{i}},$for N−1 consecutive values of i between i=1 and i=N, creating anassociation between the recorded triplet of sequences of discrete timederivatives, an identifier of a respective pre-determined gesture, andan identifier of a pre-determined computer-executable action, andstoring the recorded triplet of sequences of discrete time derivatives,the identifier of the respective pre-determined gesture, the identifierof the pre-determined computer-executable action, and the createdassociation.
 28. The wearable HMD of claim 22, wherein the threeorthogonal measurement axes comprise a first triplet of orthogonalmeasurement axes of the magnetometer device, wherein the magnetometerdevice comprises a second triplet of orthogonal measurement axes,wherein operating in the background detection state further comprisescarrying out further functions of the background detection stateincluding: measuring three orthogonal components of the backgroundmagnetic field with the second triplet of orthogonal axes to determine asupplementary field magnitude of the background magnetic field,determining an occurrence of a supplementary trigger from the HWM bydetecting a supplementary perturbation by the HWM of the determinedsupplementary field magnitude at least as large as a supplementaryperturbation threshold, and upon determining the occurrence of at leastone of the trigger and the supplementary trigger, transitioning tooperating in the gesture detection state; and wherein operating in thegesture detection state further comprises carrying out further functionsof the gesture detection state including: tracking motion of the HWM bydetermining supplementary time derivatives of magnetic field strengthmeasured with the second triplet of orthogonal axes, making a jointcomparison of the determined time derivatives and the determinedsupplementary time derivatives with one or more combined sets ofpre-determined time derivatives and pre-determined supplementary timederivatives, each of the one or more combined sets being stored at thewearable HMD and each being associated with one of the respective knowngestures, upon jointly matching both the determined time derivatives andthe determined supplementary time derivatives with a particular combinedset of the one or more combined sets based on the joint comparison,identifying the respective known gesture associated with the particularcombined set.
 29. A nontransitory computer-readable medium havinginstructions stored thereon that, upon execution by one or moreprocessors of a wearable head-mounted display (HMD), cause the wearableHMD to carry out operations comprising: operating in a backgrounddetection state; while operating in the background detection state,carrying out functions of the background state including, measuringthree orthogonal components of a background magnetic field using threeorthogonal measurement axes of magnetometer device of the wearable HMD,and determining a field magnitude of the background magnetic field fromthe three measured orthogonal components, determining an occurrence of atrigger from a hand-wearable magnet (HWM) at a time T_(start) upondetecting a perturbation by the HWM of the determined field magnitude atleast as large as a perturbation threshold, and upon determining theoccurrence of the trigger, transitioning to operating in a gesturedetection state; and while operating in the gesture detection state,carrying out functions of the gesture detection state including, for theduration of the gesture detection state, tracking motion of the HWM bydetermining time derivatives of magnetic field strength measured by themagnetometer device along each of the three orthogonal measurement axes,making a comparison of the determined time derivatives with one or moresets of pre-determined time derivatives of magnetic field strength,wherein each of the one or more sets is configured to be stored at thewearable HMD and each is associated with a respective known gesture,upon matching the determined time derivatives with a particular set ofthe one or more sets based on the comparison, identifying the respectiveknown gesture associated with the particular set, and transitioning tooperating in the background detection state upon both of, measuring themagnitude perturbation by the HWM of the determined field magnitude tobe less than the perturbation threshold, and determining an expirationof a time interval W that begins at T_(start).
 30. The nontransitorycomputer-readable medium of claim 29, wherein the instructions, uponexecution by one or more processors of the wearable HMD, cause thewearable HMD to carry out further operations comprising: responsive toidentifying the respective known gesture associated with the particularset, identifying a pre-determined computer-executable action associatedwith the identified respective known gesture; and carrying out theidentified pre-determined computer-executable action.
 31. Thenontransitory computer-readable medium of claim 29, wherein detectingthe perturbation by the HWM of the determined field magnitude at leastas large as the perturbation threshold comprises determining that theperturbation by the HWM of the determined field magnitude remains atleast as large as the perturbation threshold for at least a firstthreshold length of time, and wherein measuring the magnitudeperturbation by the HWM of the determined field magnitude to be lessthan the perturbation threshold comprises determining that theperturbation by the HWM of the determined field magnitude remains lessthan the perturbation threshold for at least a second threshold lengthof time.
 32. The nontransitory computer-readable medium of claim 29,wherein measuring the three orthogonal components of a backgroundmagnetic field comprises measuring magnetic field components H_(x),H_(y), and H_(z) of a magnetic field vector {right arrow over(H)}=[H_(x), H_(y), H_(z)] at consecutive discrete times t_(i), i=1, . .. , N₂, over a sliding time window W_(N) ₁ _(,N) ₂ from N₁ to N₂,wherein N₂=N₁+N−1 and N≧1, wherein determining the field magnitude ofthe background magnetic field from the three measured orthogonalcomponents comprises: computing magnetic field magnitude samplesh_(i)=∥H∥=√{square root over (H_(x) ²+H_(y) ²+H_(z) ²)} at theconsecutive discrete times t_(i), i=N₁, . . . , N₂ over W_(N) ₁ _(,N) ₂; and computing a sample mean field strength μ_(N) ₁ _(,N) ₂ =1/NΣ_(i=N)₁ ^(N) ² h_(i) over W_(N) ₁ _(,N) ₂ , and wherein detecting theperturbation by the HWM of the determined field magnitude at least aslarge as the perturbation threshold comprises making a determinationthat at least one of the field magnitude samples h_(j) deviates fromμ_(N) ₁ _(,N) ₂ by an amount at least as large as the perturbationthreshold, wherein N₁≦j≦N₂.
 33. The nontransitory computer-readablemedium of claim 32, wherein making the determination that the at leastone of the field magnitude samples h_(j) deviates from μ_(N) ₁ _(,N) ₂by an amount at least as large as the perturbation threshold comprisesboth of: determining that the at least one of the field magnitudesamples h_(j) deviates from μ_(N) ₁ _(,N) ₂ by an amount at least aslarge as the perturbation threshold; and concurrently determining that amotion detector of the wearable HMD is detecting motion at no greaterthan a threshold level of motion.
 34. The nontransitorycomputer-readable medium of claim 29, wherein each of the one or moresets of pre-determined time derivatives of magnetic field strengthcomprises a respective triplet of pre-determined sequences of discretetime derivatives of a magnetic field along the three orthogonalmeasurement axes, wherein determining the time derivatives of magneticfield strength measured by the magnetometer device along each of thethree orthogonal measurement axes comprises: during at least a portionof the time interval W, measuring orthogonal magnetic field componentsH_(x), H_(y), and H_(z) of a magnetic field vector {right arrow over(H)}=[H_(x), H_(y), H_(z)] at consecutive discrete times t_(i), i=1, . .. , N, wherein N≧2, to obtain samples of the of H_(x), H_(y), and H_(z)at the consecutive discrete times t_(i), i=1, . . . , N; and computingdiscrete time derivatives of the measured orthogonal magnetic fieldcomponents H_(x), H_(y), and H_(z) from discrete differences betweensuccessive samples, and wherein matching the determined time derivativeswith the particular set of the one or more sets based on the comparisoncomprises determining a closest match between the computed discrete timederivatives and the respective triplet of pre-determined sequences ofdiscrete time derivatives of one of the one or more sets.
 35. Thenontransitory computer-readable medium of claim 29, wherein each of theone or more sets of pre-determined time derivatives of magnetic fieldstrength comprises a respective triplet of pre-determined sequences ofdiscrete time derivatives of a magnetic field along the three orthogonalmeasurement axes, and wherein the instructions, upon execution by one ormore processors of the wearable HMD, cause the wearable HMD to carry outfurther operations comprising: receiving a recording command, andresponsively transitioning to operating in a gesture recording state;and while operating in the gesture recording state, carrying outfunctions of the gesture recording state including, determining anoccurrence of a recording trigger from the HWM by detecting aperturbation by the HWM of the determined field magnitude at least aslarge as a recording perturbation threshold, upon determining theoccurrence of the recording trigger, measuring orthogonal magnetic fieldcomponents H_(x), H_(y), and H_(z) of a magnetic field vector {rightarrow over (H)}=[H_(x), H_(y), H_(z)] at consecutive discrete timest_(i), i=1, . . . , N, wherein N≧2, to obtain samples of the of H_(x),H_(y), and H_(z) at the consecutive discrete times t_(i), i=1, . . . ,N, computing discrete time derivatives of the measured orthogonalmagnetic field components H_(x), H_(y), and H_(z) from discretedifferences between successive samples to obtain a recorded triplet ofsequences of discrete time derivatives$\left( \frac{\mathbb{d}H_{x}}{\mathbb{d}t} \right)_{i},\left( \frac{\mathbb{d}H_{y}}{\mathbb{d}t} \right)_{i},{{and}\mspace{14mu}\left( \frac{\mathbb{d}H_{z}}{\mathbb{d}t} \right)_{i}},$for N−1 consecutive values of i between i=1 and i=N, creating anassociation between the recorded triplet of sequences of discrete timederivatives, an identifier of a respective pre-determined gesture, andan identifier of a pre-determined computer-executable action, andstoring the recorded triplet of sequences of discrete time derivatives,the identifier of the respective pre-determined gesture, the identifierof the pre-determined computer-executable action, and the createdassociation.